COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Report from the Commission to the Council and to the European Parliament for the interim evaluation of the Programme for the Competitiveness of Enterprises and Small and Medium-sized enterprises
Date: 2019-10-14
Institutions: Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs, European Commission
Subjects: EU programme, business start-up, company growth, competitiveness, corporate finance, market access, organisational culture, single market, small and medium-sized enterprises, sustainable development

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 14.10.2019  SWD(2019) 374 final  COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Report from the Commission to the Council and to the European Parliament for the interim evaluation of the Programme for the Competitiveness of Enterprises and Small and Medium-sized enterprises {COM(2019) 468 final}  
 
 
COMMISSION STAFF WORKING DOCUMENT 

EVALUATION 

Accompanying the document 

Report from the Commission to the Council and to the European Parliament 

for the interim evaluation of the Programme for the Competitiveness of Enterprises and 
Small and Medium-sized enterprises 

1 

 
 
1. 

INTRODUCTION ................................................................................................................................ 4 

Purpose and scope ...................................................................................................................... 4 

2. 

BACKGROUND TO THE INTERVENTION ..................................................................................... 5 

Description of the intervention and its objectives ...................................................................... 5 

3. 

IMPLEMENTATION / STATE OF PLAY .......................................................................................... 8 

Description of the current situation ............................................................................................ 8 

4.  METHOD............................................................................................................................................ 14 

4.1.  SHORT DESCRIPTION OF METHODOLOGY ............................................................................... 14 

4.2.  LIMITATIONS AND ROBUSTNESS OF FINDINGS ..................................................................... 16 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 18 

5.1.  RELEVANCE ..................................................................................................................................... 18 

5.2.  EFFECTIVENESS .............................................................................................................................. 24 

5.3.  EFFICIENCY...................................................................................................................................... 29 

5.4.  COHERENCE ..................................................................................................................................... 32 

5.5.  EU ADDED VALUE .......................................................................................................................... 35 

6. 

CONCLUSIONS AND LESSONS LEARNED ................................................................................. 39 

ANNEX 1: PROCEDURAL INFORMATION 

ANNEX 2: STAKEHOLDER CONSULTATION 

ANNEX 3: METHODS AND ANALYTICAL MODELS 

ANNEX  4:  INTERIM  EVALUATION  OF  THE  INTERNATIONAL  INTELLECTUAL 

PROPERTY RIGHTS (IPR) SME HELPDESK, A SPECIFIC COSME MEASURE 

2 

 
 
 
Glossary 

Term or acronym 

Meaning or definition 

COSME 

The Programme for the Competitiveness of Enterprises and small and medium-sized 
enterprises (COSME) (2014 - 2020) 

SME 

Small and medium-sized enterprises 

CIP 

EIP 

MFF 

EIF 

The Competitiveness and Innovation Programme (CIP) (2007-2013) 

The Entrepreneurship and Innovation Programme (EIP) (2007-2013) 

Multi-annual Financial Framework 

The European Investment Fund 

EASME 

The Executive Agency for SMEs 

LGF 

EFG 

EEN 

EYE 

Loan Guarantee Facility 

Equity Facility for Growth 

The Enterprise Europe Network 

The Erasmus for Young Entrepreneurs 

IPR SME Helpdesk 

Intellectual Property rights for SMEs Helpdesks 

IPR 

EFSI 

SO 

MS 

WP 

EU 

R&I 

KETs 

FC 

GA 

IOs 

TA 

OPC 

SBA 

Intellectual Property rights 

The European Fund for Strategic Investments  

Specific Objective 

Member States 

Work Programme 

European Union 

Research and Innovation 

Key Enabling Technologies 

Framework Conditions in the Single Market 

Grant agreements 

Intermediary Organisations 

Thematic Area 

Open Public Consultation 

Small Business Act  

3 

 
 
1. 

INTRODUCTION 

Purpose and scope 

This  evaluation  assesses  the  relevance,  effectiveness,  coherence,  efficiency  and  EU 
added-value of the Programme for the Competitiveness of Enterprises and small and 
medium-sized enterprises (COSME) (2014  - 2020) from 2014 to 2016, in conformity 
with article 15.3 of its basic act1:  

 “By 2018 at the latest, the Commission shall establish an interim evaluation report on 
the  achievement  of  the  objectives  of  all  the  actions  supported  under  the  COSME 
programme at the level of results and impacts, the efficiency of the use of resources and 
its  European  added  value,  with  a  view  to  a  decision  on  the  renewal,  modification  or 
suspension of the measures. The interim evaluation report shall also address the scope for 
simplification,  its  internal  and  external  coherence,  the  continued  relevance  of  all 
objectives,  as  well as  the contribution  of the measures to  the Union priorities of smart, 
sustainable and inclusive growth. It shall take into account evaluation results on the long-
term  impact  of  the  predecessor  measures  and  shall  feed  into  a  decision  on  a  possible 
renewal, modification or suspension of a subsequent measure.” 

The interim evaluation contributed to the preparation of the future programme in the next 
Multi-annual Financial Framework for the period 2021-2027. 

The performance of the delegated entities (The European Investment Fund (EIF) and the 
Executive Agency  for SMEs (EASME) is  an important  aspect of the assessment  of the 
efficiency and effectiveness criteria and for addressing the scope for simplification. This 
evaluation  will  feed  into  the  mid-term  evaluation  of  EASME  that  will  analyse  in  more 
depth the performance of the Agency in implementing the programmes delegated to it. 

The  evaluation  covers  the  28  Member  States  of  the  European  Union  and  the  COSME 
participating countries. The article 6 of the COSME regulation allows third countries to 
participate  in  the  programme.  In  2016,  the  following  countries  participate  in  COSME: 
Albania,  Montenegro,  the  Republic  of  North  Macedonia,  Iceland,  Turkey,  Serbia, 
Moldova  and  Armenia2.  Ukraine3  and  Bosnia  signed  the  international  agreement  for 
participation to COSME in respectively May and June 2016, pending ratification by the 
national  parliaments4,  but  these  agreements  were  not  in  force  during  the  period 
contemplated by this evaluation. 

1 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32013R1287 

2 Moldova, Armenia and Ukraine will not participate to the COSME financial instruments. 

3 See footnote 10 

4 In addition, for the Enterprise Europe Network, possibilities exist for business support organisation in non-
participating countries to associate themselves with the Network. They will not receive funding from the programme 
but should be entitled to use tools and information developed within the Network. 

4 

 
 
                                                           
2.  BACKGROUND TO THE INTERVENTION 

Description of the intervention and its objectives 

The Programme for the Competitiveness of Enterprises and small and medium-sized 
enterprises  (COSME)  (2014  -  2020)5  is  the  successor  of  the  Competitiveness  and 
Innovation  Programme  (CIP)  (2007-2013)6.  With  a  total  budget  of  EUR  2.3  billion, 
COSME  aims  to  contribute  to  the  Commission’s  top  political  priority  of  creating  more 
growth and jobs.  It also aims to support the creation and growth of small and medium-
the  competitiveness  of  EU  enterprises  and 
sized  enterprises,  and  promote 
entrepreneurship. 

SMEs  are  the  main  source  of  economic  growth  and  job  creation  in  the  EU.  They 
constitute 99% of European businesses, provide two out of three private sector jobs and 
contribute more than half of the total value-added created by businesses in the EU. The 
impact  assessment  accompanying  the  proposal  for  the  2014-2020  COSME  programme 
identified the following main problems7 SMEs are facing across the EU:  

  Cross-border regulatory fragmentation; 
  Excessive administrative burden for starting a business; 
  Lack of entrepreneurship spirit in the EU;  
  Difficulties to access to finance; 
  Significant  barriers  when  doing  business  across  borders  inside  and  outside 

Europe.  

To  respond  to  these  market  failures  and  address  the  problem,  the  COSME  Regulation 
sets two general objectives and four specific objectives to address these problems. 

General objectives: 

1.  To  strengthen  the  competitiveness  and  sustainability  of  the  EU's  enterprises, 

particularly SMEs and 

2.  To  encourage  entrepreneurial  culture  and  promoting  the  creation  and  growth  of 

SMEs. 

Specific objectives (SOs): 

1.  (SO  1)  Easing  access  to  finance  for  SMEs,  both  through  guarantees  and  equity  (at 

least 60% of the total budget);  

2.  (SO  2)  Helping  SMEs  access  to  markets  and  internationalise  (indicative  21.5%  of 

the total budget); 

3.  (SO  3)  Creating  a  favourable  environment  for  enterprises  and  support  their 

competitiveness (indicative 11% of the total budget); 

4.  (SO 4) Promoting entrepreneurship (indicative 2.5% of the total budget). 

5 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1287 

6 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=URISERV%3An26104 

7 From the executive summary of the CWP Impact Assessment accompanying the proposal for a COSME Regulation, 
COM(2011)834 SEC(2011)1452, http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1452 

5 

 
 
 
                                                           
The  remaining  5%  of  the  total  budget  are  available  for  administrative  expenditures  in 
particular for the EASME subsidy, horizontal IT tools, corporate communication. 

The Commission delegates the implementation of COSME to the Executive Agency for 
SMEs  (EASME)8  for  all  objectives9  with  the  exception  of  the  implementation  of  the 
financial instruments  under specific objective 1  entrusted to  the  European Investment 
Fund (EIF).  

The diagram in the next page shows the intervention logic of the COSME programme:  

8 EASME is the successor of the Executive Agency for Competitiveness and Innovation (EACI) 

9 92% on average for 2014 and 2015 for objectives 2, 3 and 4. 

6 

 
  
 
                                                           
7 

ObjectivesNeeds, problems and issues•EU SMEs face difficulties to access finance•EU SMEs still face significant barriers when doing business across borders inside and outside Europe •Administrative and regulatory burden is still an issue for EU enterprises, in particular for SMEs•Lack of entrepreneurial spirit in the EUThe COSME Regulation (EU) No 1287/2013 is in line with the Europe 2020 strategy to reinforce the growth of theEuropean Economy while delivering a high level of employment, a low-carbon, resource and energy efficient economy and social cohesion. SMEs should play a crucial role in reaching the Europe 2020 strategy objectives.General objective: To support the creation and growth of EU enterprises, in particular SMEs, as well as their Competitiveness and the promotion of Entrepreneurshipthrough 4 specific objectivesInputsOutputsOutcomesImpactsResultsInstrument supporting SO 1: Loan Guarantee Facility and Equity Facility for Growth, Instruments supporting SO 2: Enterprise Europe Network+ other smaller actions like IPR SME helpdesks or Your Europe Business PortalInstrument supporting SO 3: SME Policy, Better Regulation/REFIT, Tourism, Cluster policy actions, modern. of Industry + othersmaller actionsInstruments supporting SO 4: Erasmus for young Entrepreneurs Mobility Scheme + other smaller actions SO 1: Contracts with financial intermediaries to provide loansfor SMEs and equity funds + contracts for data and information and communicationSOs 2. to 4:  Contracts and Grants for advisory services from the Enterprise Europe Network; mobility scheme for young entrepreneurs; market replication projects; support for entrepreneurial spirit & creation of start-ups, support to simplification & better regulation, Promotion & dissemination projects, support for policy reform; business opportunities, support activities, studies, information and communication.SO 1: Greater financing for SMEs & increase in productive investments by SMEs; New business start-ups; SO 2: Improved access to markets and SMEs reaping the benefits of the single marketSO 3: Improved cooperation between entrepreneurs and policy makers; a more businessfriendly regulatory and administrative Framework; SO 4: Increase of the entrepreneurial sprit to spur the creation of SMEs and start-ups.Improved competitiveness of EU SMEs; Achievement of EU 2020 targetsCreation of jobs and growthSpecific objectives (SO):1. Access to finance for SMEs2. Access to markets3. Promoting a favourable business environment  and competitiveness4. Promoting Entrepreneurship 
 
3. 

IMPLEMENTATION / STATE OF PLAY 

Description of the current situation  

three  ‘pillars’: 

COSME  is  the  successor  of  the  EIP  pillar  of  the  Competitiveness  and  Innovation 
Framework Programme (CIP). CIP ran between 2007 and 2013 with a budget envelope 
of  EUR  3.6  billion10.  Its  objectives  were  to  contribute  to  strengthening  productivity, 
innovation  capacity,  and  sustainable  growth.  It  comprised 
the 
Entrepreneurship  and  Innovation  Programme  (EIP),  the  Information  Communication 
Technologies Policy Support Programme (ICT-PSP), and the Intelligent  Energy Europe 
Programme (IEE). The EIP accounted for slightly less than 60% of the CIP budget. In the 
2014-2020 programming  period, the innovation  activities (including  ICT-PSP and  IEE) 
were  transferred  to  Horizon  2020,  while  the  activities  under  the  EIP  programme  were 
largely  transferred  to  COSME.  The  EIP  supported  access  to  finance  for  SMEs,  the 
Enterprise Europe Network (established in 2008 by integrating two former networks: the 
Euro Info Centres and the Innovation Relay Centres, Erasmus for Young Entrepreneurs, 
the  IPR Helpdesks for SMEs, and activities aimed at  fostering  entrepreneurship  culture 
and creating better framework conditions for SMEs operating in the EU. It also provided 
actions  to  support  innovation  in  enterprises  (including  eco-innovation)  and  innovation 
governance and culture.  

While EIP under CIP strived ‘to support entrepreneurship and innovation and to promote 
the  development  and  growth  of  SMEs  across  the  EU’,  COSME  takes  a  more  targeted 
approach and aims at improving the business environment to enhance competitiveness by 
focusing  on  framework  conditions,  entrepreneurship,  access 
to  finance  and/or 
markets. All  references to  innovation  are left out in  COSME, while special  attention to 
tourism has been introduced.11. 

In  most  cases,  the  specific  actions  that  were  implemented  or  piloted  in  the  EIP 
programme  are  continued  also  in  COSME,  with  only  minor  changes.  More  substantial 
changes have been introduced in the financial instruments and the EEN services, listed in 
Table 1, below.  

Table 1: The financial instruments and EEN services in CIP versus COSME 

Focus area 

Under EIP 

Under COSME 

Financial 
instruments 

Three financial instruments: 
  the High Growth and Innovative 

SME Facility (GIF)  

  the SME Guarantee Facility 

(SMEG), and  

  the Capacity Building Scheme 

(CBS) 

EEN 

Key focus is on facilitating 
cooperation among SMEs 

Two financial instruments: 
  EFG was created as a translation of GIF2 with 
adjusted terms and conditions of investment 

  The LGF is a direct translation of the SMEG Windows 

on loans, microcredit and securitization 

  The CBS was discontinued because of a low take up 

  Key focus is on facilitating access to markets and 

internationalization (within the EU) 
  More customized services to SMEs 
  Expansion of the online services 
  Advisory services as an additional service 

Source: Technopolis Group, based on CIP/EIP and COSME programme documents 

10 COM(2005) 121  

11 For a detailed comparison of CIP and COSME: Fanny Lajarthe (2012). Differences and Similarities between CIP 
and COSME: Briefing Note. IP/A/ITRE/2012-14 

8 

 
 
 
                                                           
The  final  CIP  evaluation12  concluded  that  the  programme  objectives  related  directly  to 
the  needs,  problems  and  issues  they  were  intended  to  address.  In  addition,  the 
concentration  of  actions  on  areas  where  EU  action  could  make  a  difference  had  been 
identified  as  a  positive  shift.  CIP  also  proved  to  be  flexible  and  able  to  respond  to 
emerging  issues.  However,  stakeholders  raised  the  issue  of  fragmentation  due  to  the 
financing of many small activities and the evaluation recommended a continued need to 
justify  each  small-scale  measure  separately,  a  need  to  strengthen  mechanisms  for  the 
overall  coordination  of  actions  (to  avoid  unnecessary  proliferation  and  achieve  greater 
synergies),  and  a  need  to  improve  the  monitoring  and  communication  of  the  aims  and 
achievements  of  small  scale  measures.  It  pointed  out  that  in  general,  the  indicators 
chosen to monitor the implementation and results of the actions were not all responding 
to the SMART criteria. Questions were also raised concerning synergies across the three 
components  of  CIP,  and  issues  were  spotted  in  relation  to  the  coherence  of  the 
programme, amongst which the coherence of the Financial Instruments with the support 
offered by the Structural Funds.  

The need to further develop and improve the monitoring system was a major finding in 
the  final  EIP  evaluation,  which  recommended  the  development  of  “a  clear  overall 
monitoring  system  that  regularly  provides  data  in  an  easily  accessible  and  standard 
format”.13  The  evaluation  team  noted  that  there  was  a  very  limited  synergy  or  overlap 
with financial instruments on a national level and that the various financial instruments of 
the Commission were still seen to be fragmented and overlapping. Both the interim and 
final evaluation of the EIP14 highlighted that there was some sort of competition between 
venture  capital  and  loan  guarantee  schemes  under  the  ERDF  and  under  the  EIP 
programme. The EIP final evaluation considered that the demand-driven approach to the 
financial  instruments  could  give  way  to  one  that  is  more  pro-active  and  especially  one 
that  concentrates  on  promoting  facilities  in  countries  not  yet  covered. Finally,  the  EIP 
evaluation  raised  the  danger  of  having  too  many  small  activities  funded  by  the  EIP 
programme  and  noted  the  absence  of  “a  mechanism  that  could  ensure  greater 
coordination  and  synergies  among  the  different  activities”.  It  proposed  to  establish  an 
enhanced co-ordination function in the management of the programme that would have 
the SBA as a major reference point and focus on “an active exploitation of the potential 
synergies, both within the Programme and with external actions”.  

The  COSME  programme  is  implemented  through  annual  work  programmes,  which  are 
set  up  according  to  the  Commission's  policy  priorities  and  approved  through  the 
examination procedure by the Member States' Committee.  

The budget execution rates of COSME are good. . The operational budget15 available in 
2014  (EUR  247  million),  was  executed  at  99.87%;  in  2015  the  operational    budget 
available  (EUR  329  million)  was  executed  at  99.38%;  in  2016  the  operational  budget 
available (EUR 336 million) was executed at 96.69%. The implementation structures for 

12 CSES (2011). Final Evaluation of the Competitiveness and Innovation Framework Programme 

13 Final Evaluation of the Entrepreneurship and Innovation Programme - Final Report, Centre for Strategy and 
Evaluation Services, April 2011  

14 Interim evaluation of the Entrepreneurship and Innovation Programme (April 2009) and Final evaluation of the 
Entrepreneurship and Innovation Programme (April 2011) 

15 Budget lines 02.0201 and 02.0202 for all fund sources, including third countries contributions and reflows from the 

CIP financial instruments and not including the COSME administrative budget lines. 

9 

 
 
 
                                                           
the  financial  instruments  and  the  Enterprise  Europe  Network  were  in  place  and 
delivering.  The  total  authorized  appropriations  for  2016  amount  to  EUR  298  million. 
(There  was  a  further  reinforcement  of  EUR  550  million  of  the  financial  instruments 
through EFSI (see section below).  From 2016, the programme has integrated priorities 
of  the  Single  market  strategy,  particularly  the  priority  to  support  SMEs,  start-ups  and 
scaling-ups.  

Access to Finance - specific objective 1 

60% of the Programme’s  resources  are allocated to  the COSME  Financial  Instruments, 
the Loan Guarantee Facility (LGF) and the Equity Facility for Growth (EFG). They build 
on  the  success  of  the  CIP  Financial  Instruments  (2007-2013)  which  helped  to  mobilise 
more than EUR 20 billion of loans and EUR 3 billion of venture capital to over 380,000 
SMEs  in  Europe.  The  COSME  financial  instruments  operate  in  conjunction  with  the 
financial  instruments  set  up  under  Horizon  2020  as  part  of  a  single  debt  financial 
instrument and a single equity financial instrument respectively, supporting the growth of 
enterprises and their R&I activities. They can also complement financial instruments set 
up under the European Structural and Investment Funds. 

The calls for expression of interest for the financial instruments published in 2014 have 
generated  a  strong  interest  from  financial  intermediaries  across  the  EU  and  non-EU 
Member  State  countries  participating  in  COSME,  especially  for  the  LGF.  The  strong 
market  demand  for  the  LGF  continued  in  2015,  with  the  consequence  that  the  whole 
2014-2015  budget  available  was  exhausted  by  mid-2015.  Thanks  to  the  guarantee 
provided  by  the  European  Fund  for  Strategic  Investment  (EFSI)16  the  EIF  could 
continue to sign COSME LGF guarantee agreements in the second half of 2015. At 
the end of 2016, almost  143 000 SMEs in 21 participating countries received financing 
for almost EUR 5.5 billion17 under the enhanced LGF. The COSME LGF enhancement 
thanks  to  the  EFSI  guarantee  continued  in  2016  and  2017,  with  a  positive  effect  on 
financing  made  available  to  riskier  SMEs  at  a  much  earlier  moment  in  time  as  would 
have been the case with the COSME budget only, thereby speeding up the financing of 
the real economy.  

Given  the  specific  features  inherent  to  an  equity  facility,  with  more  complex  due 
diligence and fund raising processes, signature of fund agreements took more time. The 
first  five  transactions  under  EFG,  out  of  which  three  multi-stage  funds  in  combination 
with the InnovFin Equity Facility for Early Stage set up under Horizon 2020, have been 
signed  end  of  2015.  First  investments  into  SMEs  took  place  in  2016  under  these 
agreements. 

The total budget foreseen for the financial instruments under the 2016 work programme 
(EUR 295.3 million) was fully implemented. 

Under  specific  objective  1,  COSME  also  finances  surveys  to  measure  the  access  to 
finance  situation  in  Europe18,  awareness  raising  means  about  the  EU  financial 

16 The so-called COSME LGF enhancement 

17 See COSME LGF Implementation Status as at 31 March 2016 available at: 
http://www.eif.org/what_we_do/guarantees/single_eu_debt_instrument/cosme-loan-facility-growth/index.htm 

18 SAFE survey: http://ec.europa.eu/growth/access-to-finance/data-surveys_en 

10 

 
 
 
   
 
  
 
                                                           
instruments19,  as  well  as  other  support  actions  on  access  to  finance  for  about  EUR  1 
million each year. 

Access to markets – specific objective 2 

COSME  helps  European  enterprises  and,  in  particular  SMEs,  to  do  business  in  the 
Single market as well as to export outside Europe. 

More  than  two  thirds  of  the  COSME  budget  for  access  to  markets  is  devoted  to  the 
Enterprise Europe Network (EEN), which helps SMEs to find business, technology and 
research  partners  in  the  EU  and  third  countries.  It  also  promotes  the  participation  of 
SMEs  in  EU  funding  programmes  such  as  Horizon  2020  and  the  European  Investment 
and Structural Funds. 

The  EEN  started  its  activities  as  planned  on  1/1/2015  (the  activities  in  2014  were  still 
covered by the CIP programme).  92 consortia were selected as a result of two calls for 
proposals.  The  Network  currently  includes  525  organisations  in  the  EU  and  80 
organisations  in  the  eight  COSME  participating  countries;  they  signed  framework 
partnership  agreements  covering  a  seven  year  period  and  specific  grant  agreements  for 
the operational periods 2015-2016 and 2017-2018. 

The Network (about 3.000 staff members) involved, on yearly basis, more than 200.000 
SMEs in local events, provided specialised advisory services to 70.000 SMEs on access 
to finance, IPR business and technology, resource efficiency services etc.  About 25.000 
SMEs  participate  in  matchmaking  events  every  year  resulting  in  about  2.500 
international partnerships signed between SME. 

Starting from July 2017, the EEN is implementing an action focused on start-ups/scaling 
up, a priority of the Single Market Strategy, for a budget of EUR 2.6 million.  The EEN 
scale-up advisors will help young European SMEs scaling up their activities in the Single 
Market by training, mentoring, coaching and innovation and internationalisation services. 

Under specific objective 2 COSME also funds several other smaller projects; the most 
important are: 

  The Your Europe Business portal for about EUR 0,5 million per year; 
  The intellectual property rights (IPR) helpdesks in China, Latin America and south east Asia for 

EUR 7,2 million; 

  The EU Japan Centre for Industrial Cooperation EUR 2,7 million per year; 
 
 

Improved points of single contacts in the Member States for EUR 1 million; 
Improving access to SMEs in public procurement, EUR 1 million. 

Improve framework conditions for businesses – specific objective 3 

COSME funds actions aiming at facilitating SMEs' access to IPR, public procurement, 
clusters and digital technologies, cutting red tape and promoting the "Think Small 
First" principle and improving the competitiveness of strategic sectors which have a 
strategic importance for the European economy including, tourism.  

19 EU Access to Finance web portal: http://europa.eu/youreurope/business/funding-grants/access-to-finance/ 

11 

 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
The  Commission  implements  this  objective  through  a  wide  variety  of  smaller  actions, 
through calls for proposals and calls for tenders such as: 

  The E-Skills for competitiveness and Innovation, about EUR 3 million per year; 
  The Cluster  internationalization program for SMEs, ranging from EUR 3,5 to 5 million per year; 
  The Cluster Excellence programme for EUR 1,5 million per year; 
  Modernization of Industry activities (Key Enabling Technologies, Advanced Manufacturing, etc.) 

for EUR 5 million per year on average; 

  Several  activities  to  support  SMEs  in  industrial  sectors  (Construction,  Chemistry,  Aeronautics, 
Defense,  the  food  supply  chain,  Resource  Efficiency,  Design,  Bio  Based  Products,  …);  ranging 
from EUR 5 to 8 million per year; 

  The monitoring and follow-up of the SME policy, for EUR 5 million per year on average: 

o  The SME Performance Review;  
o  The SME assembly; 
o  The SME Envoys Network; 
o  The outreach tools like Business Planet on Euronews and the SME portal; 
  EU REFIT platform for regulatory burden reduction, EUR 0,5 million on average per year 
  The support to Tourism sector for a budget ranging from EUR 5 to 9 million per year; 
  The promotion of the social economy and social entrepreneurship, for a budget ranging from EUR 

0.5 to EUR 2 million per year. 

Between  2014  and  2016,  EUR  109.3  million  were  allocated  to  this  objective, 
representing  12%  of  the  financial  envelope.  The  first  calls  under  this  objective  were 
published at the end of 2014 and considering the period of execution, most of the actions 
only started producing results after the time scope for this evaluation.   

The  High-level  Group  on  administrative  burden  contributed  to  the  exchange  of  good 
practices and the follow-up up and paved the way for the EU REFIT platform. The SME 
Performance  Review  provided  an  overview  of  the  European  SMEs  and  detailed 
information  on  the  implementation  of  the  Small  Business  Act.  The  Member  States-
specific 
the  European  Semester  country  specific 
recommendations.  The  network  of  SME  Envoys,  the  annual  SME  Assembly  and  the 
European SME week provided fora for awareness raising and exchange of good practices 
in  all  areas  of  the  Small  Business  Act.  Last  but  not  least,  the  4th  season  of  business 
planet on Euronews TV started on 12 June 2015.  

information  was 

into 

fed 

The  e-skills  for  jobs  campaign  covered  30  countries,  involving  384  organisations  and 
gathering more than 300.000 participants  with  some 115 million  people exposed to  the 
campaign. 

The call for proposals "Design based consumer goods" (with an available budget of EUR 
4 million) resulted in 10 SME projects awarded to develop innovative, digital solutions to 
improve  design  processes  and  communication  in  the  textile  and  clothing  value  chain, 
innovative products embedding digital technologies to products and new business models 
addressing environmental and societal challenges.  

The tourism initiatives contributed to strengthening the competitiveness of the European 
tourism  sector.  The  initiatives  dedicated  to  the  increase  of  senior  and  young  tourists' 
flows  contributed  to  generate  an  extension  of  the  tourism  season.  Actions  to  enhance 
tourism  accessibility  contributed  to  improving  facilities  and  services  for  tourists  with 
special  access  needs.  The  visibility  of  European  Destinations  of  Excellence  has  been 
improved, which will trigger higher numbers of potential tourist arrivals and local spill-
over  revenues  for  the  destinations  and  tourism  operators.  Awareness-raising  activities 
and events generated a better visibility and up-take of tourism initiatives.  

12 

 
 
 
 
 
 
 
 
In 2014-2016, the Clusters Go International calls involved 25 European Strategic Cluster 
Partnerships  (ESCPs),  representing  145  cluster  organisations  across  23  European 
countries and more than 17,000 European SMEs. 15 out of the 25 selected ESCPs were 
funded.  The  other  10  partnerships  were  awarded  the  ESCP  label  and  encouraged  to 
continue  their  partnerships  on  a  voluntary  basis,  without  COSME  funding.  The  15  co-
funded  partnerships  account  for  approximately  95  clusters  reaching  out  to  over  10,300 
European SMEs.  

WATIFY  is  an  awareness  raising  campaign  launched  by  the  European  Commission  in 
2014  seeking  to  promote  digital  entrepreneurship  and  modernisation  of  the  European 
industry. Implemented in two distinct phases, the campaign sought to reach SMEs, public 
authorities and other intermediaries across Europe and inform them about opportunities 
brought by digitalisation. 

Entrepreneurship and Entrepreneurial culture – specific objective 4 

COSME  supports  exchanges  among  European  educators  and  trainers  to  develop  best 
practices  in  entrepreneurship  education  in  the  EU.  The  programme  focuses  on  groups 
such as young people, women or senior entrepreneurs in order to benefit from mentoring 
or other tailored schemes.  

The main tool is the Erasmus for Young Entrepreneurs mentoring programme, which 
helps new entrepreneurs acquire and build entrepreneurial skills and knowledge, and to 
further  develop  their  business  activity  by  learning  from  experienced  entrepreneurs.  It 
started in 2009 as a pilot and is now a key action in the COSME programme. 

It is implemented through yearly call for proposals resulting in the signature by EASME 
of  grant  agreements  (GA)  for  24  month  periods.  In  general  there  are  always  a  high 
number of good projects that cannot be financed because of a lack of funding.  

For the Call 2014 (cycle 7): 111 proposals submitted, 15 projects selected involving 111 
organizations from 28 COSME countries; 40 projects for a volume of approx. 14 million 
EUR could not be funded due to budget constraints For the 2015 call, 105 proposals were 
submitted, with 17 proposals selected involving 128 organisations.  In 2014 the original 
budget  of  EUR  5,5  million    was  reinforced  by  nearly  one  million  to  award  more 
proposals.    In  2015,  the  budget  was  increased  up  to  EUR  7,8  million  and  in  2016  it  is 
intended  to  increase  it  up  to  at  least  EUR  8,7  million.  The  implementation  of  the 
activities started on 1/2/2015 with network of 175 Intermediary Organisations (IOs) in 32 
countries (26 Member States and 6 COSME countries).  

By  the  end  of  2016,  11.000  entrepreneurs’  profiles  were  accepted  and  4.200  matches 
involving over 8.400 entrepreneurs were established.  

Under this specific objective COSME also funds several other smaller projects; the most 
important are: 

  Entrepreneurship education from EUR 0,4 to EUR 1 million on average per year; 
  Women’s entrepreneurship, for EUR 0,5 to EUR 1 million on average per year; 
  Senior’s entrepreneurship for EUR 0,5 to EUR 1 million on average per year; 
  Digital Entrepreneurship, EUR 2,5 million in 2015; 
  Support to Migrant Entrepreneurship, EUR 1,5 million per year. 

13 

 
 
 
 
4.  METHOD 

4.1. 

SHORT DESCRIPTION OF METHODOLOGY 

This Staff Working Document is supported by three external studies, one for the whole 
programme20 and two separate evaluations of the IPR SME helpdesks21 and WATIFY.22. 
The methodological framework for this evaluation builds upon the evaluation criteria of 
relevance,  efficiency,  effectiveness,  coherence  and  EU  added-value.  These  evaluation 
criteria assess the relationships between the various components of the intervention logic 
presented on p.9. Taking account that this is an interim evaluation, most of the expected 
mid-term results are not yet measurable. Therefore, the primary focus was on the extent 
to  which  the  programme  sets  the  building  blocks  for  reaching  the  intended  mid-  and 
long-term results and impacts. This implies an emphasis on the adequacy and quality of 
the  programme  design,  i.e.  the  formulated  objectives,  the  invested  resources,  and  the 
activities implemented – in other words, the relevance and coherence criteria.  

In  terms  of  effectiveness,  the  assessment  focused  on  the  attainment  of  the  expected 
outputs and short-term outcomes, and the enabling factors or barriers for the attainment 
of the outcomes and expected results, such as the capacity of the programme to reach and 
involve the stakeholders needed. 

The  evaluation  covers  all  actions  of  the  programme  with  a  sense  of  proportionality  for 
the four specific objectives of COSME.  

The evaluation focusses on the main actions of the COSME Programme: 

  The financial instruments: Loan Guarantee Facility and Equity Facility for Growth; 
  The Enterprise Europe network, for which an evaluation was already performed for 
the  2008-2014  period,  adding  the  two  years  of  activity  (2015-2016)  under  the 
COSME programme; 

  The Erasmus for Young Entrepreneurs, with the support of an existing study. 

Together,  these  instruments  represent  more  than  80%  of  the  budget  of  the  programme 
and constitute the principal actions for three of the four specific objectives. 

The  remaining  20%  of  COSME  budget  is  implemented  through  a  variety  of  smaller 
actions  across  the  four  specific  objectives.  In  order  to  evaluate  the  four  specific 
objectives  of  the  programme23,  the  supporting  study  contractor  was  asked  to  select  a 
sample of representative activities on the following basis:   

  The  assessment  of  the  relevance,  coherence,  and  efficiency  in  implementation 

criteria covers all actions. 

20  Interim  Evaluation  of  the  COSME  Programme,  final  report,  Technopolis,  December  2017, 

https://ec.europa.eu/docsroom/documents/28084 

21  Interim  Evaluation  of  the  International  Intellectual  Property  Rights  (IPR)  SME  Helpdesk,  a  specific 

COSME measure, Technopolis, Annex 4 of this report. 

22 Evaluation of WATIFY 1 and WATIFY 2, Centre for strategy & evaluation services, 2019. 

23 Outlined in section 2 of the present document 

14 

 
 
                                                           
  The  Assessment  against  the  effectiveness  criterion  focuses  on  the  Key  Actions, 

complemented by four case studies. 

  The European added-value and the efficiency / cost-effectiveness criteria remain 

covered through the analysis of the Key Actions only. 

The  Key  Actions  include:  the  financial  instruments  (LGF  and  EFG),  the  Enterprise 
Europe  Network  services,  the  SME  Policy  actions,  the  Cluster  Internationalisation 
programme, and the Erasmus for Young Entrepreneurs programme.  

Actions covered through case studies are: the Vulcanus scheme in the EU-Japan Centre 
action;  the  e-skills/digitisation  actions,  the  Tourism  actions  (grants),  and  the  Women 
Entrepreneurship action.  

A  mix  of  quantitative  and  qualitative  methods  for  the  evidence  collection  and  analysis 
was  used  to  allow  for  a  proper  triangulation  of  the  evidence  collected,  ensuring  the 
robustness of the findings.  

Tools and methods used in the evaluation 

Methodological approach 

Tools & methods 

Qualitative methods 

Desk research 

Interviews 

Case studies 

Workshop 

Qualitative targeted consultation 

Targeted consultation  

Public consultation 

Quantitative methods 

Statistical analyses of secondary data  

Technopolis, December 2017, https://ec.europa.eu/docsroom/documents/28084 

Number 

260+ 

120 

4 

1 

7 

7 

1 

4 

The desk research encompassed a broad range of sources, ranging from programme and 
policy  documents  to  market  studies,  impact  assessments  and  preceding  and/or  other 
evaluations with the analysis of more than 260 documents.  

120  interviews  were  organised  to  collect  background  and  context  information  on 
facilitators and barriers for the attainment of the expected effects as well as the eventual 
unintended effects.  

Four  case  studies  were  conducted  on  SO2  –  Access  to  market  (the  trainee  programme 
“Vulcanus in Japan”, run by the EU-Japan Centre), SO3 – Favourable environment (the 
Tourism  programme  and  the  E-skills  for  competitiveness  and  innovation  actions)  and 
SO4 – Entrepreneurship (the Women Entrepreneurship action).  

Seven  ‘Qualitative’  targeted  consultations  were  implemented  where  the  population  is 
lower  than  100.  However,  the  small  numbers  did  not  allow  for  meaningful  statistical 
analysis.  The  questionnaires  mostly  include  open-ended  questions,  geared  at  collecting 
qualitative information to complete the information gathered by interviews. 

Seven  ‘classical’  targeted  consultations  were  implemented  where  the  population  was 
higher  than  100,  allowing  for  meaningful  and  robust  statistical  data  analyses.  The 
questionnaires  mostly  include  close-ended  questions  and  only  a  few  open-ended 

15 

 
  
  
  
 
 
 
questions.  The  responses  were  analysed  through  descriptive  statistics,  in  several  cases 
also included cross-analyses. 

The  public  consultation24  was  a  12-week  internet-based  questionnaire.  More  than  190 
inputs and 15 position papers have been received from Business and trade associations, 
Citizens  of  the  EU,  non-profit  organisations,  Financial  intermediaries  (private  equity, 
business angels, venture capital funds, banks, promotional banks, guarantee institutions), 
Intermediary  structures  such  as  incubators/clusters/innovation  &  growth  agencies  and  
Public authorities (member states, European agencies). 

Four types of analyses were conducted on secondary data:  

  The  portfolio  analysis  entailed  the  analysis  of  the  COSME  budget  distribution 
over  the  different  structural  components  of  the  programme,  ie  the  Strategic 
Objectives  and  their  operational  (sub)objectives,  the  policy  mix  used  in  the 
programme (i.e. the policy instruments) and the modes of implementation.  

  The composition analysis focused on the stakeholders that benefit of the COSME 
programme  funding.  The  outcomes  of  this  analysis  informed  especially  the 
analysis of effectiveness, in terms of the capacity of the programme to attract the 
type  of  stakeholders  needed  for  the  attainment  of  the  desired  effects  and  the 
analysis of cost-effectiveness  

  The time-to-grant analysis, based on EASME management data, and the success 
rate analysis of the open calls for grants, based on the EASME proposal database, 
were  conducted  to  feed  into  the  assessment  of  the  COSME  efficiency  in  the 
implementation of the programme. 

  The  cost-effectiveness  analysis  (CEA)  entailed  quantifying  the  not-monetised 

benefits that would be generated by one Euro of total costs.  

The  methodology  for  the  WATIFY  study  used  desk  research,  media  and  social  media 
analysis,  surveys  (for  a  total  of  1673  responses)  and  interviews  (45)  to  collect 
triangulated data. 

4.2.  LIMITATIONS AND ROBUSTNESS OF FINDINGS  

The supporting study25, despite the limitations presented below, provided robust evidence 
in relation to the five evaluation criteria.  It allows to take stock of progress made since 
the beginning of the programme , it evidences what works well and what doesn't and has 
implemented sufficient mitigating measures to contrast the below identified limitations. 

Data availability and reliability of the data analyses 
The  COSME  regulation  foresees  the  publication  of  a  yearly  monitoring  report.    At  the 
time of the evaluation, in 2017, only the 2014 monitoring report was published, the 2015 

24 https://ec.europa.eu/info/consultations/interim-evaluation-programme-competitiveness-enterprises-and-small-and-
medium-sized-enterprises-cosme-2014-2020_en 

25 https://ec.europa.eu/docsroom/documents/28084 

16 

 
 
                                                           
report  was  only  available  in  a  draft  version  and  the  2016  data  was  partially  available.  
Besides,  the  variety  of  implementation  modes  (financial  instruments,  contracts,  grants) 
and the diversity of actions implied different data sources and there is not a consolidated 
source of data besides the monitoring report.   

On  top  of  that,  except  for  access  to  finance,  the  enterprise  Europe  network  or  the 
Erasmus for young entrepreneurs that are long-standing activities inherited from the CIP 
programmes, most COSME actions were ongoing and not finalised. This represent about 
20%  of  the  programme  and  a  substantial  part  of  the  objective  3,  framework  conditions 
and  competitiveness  and  a  less  substantial  part  of  access  to  markets  and  of 
entrepreneurship. 

Consequently,  a  major  challenge  for  this  evaluation  was  the  difficulty  to  obtain  the 
needed  data  and  the  quality  of  the  data  provided.  While  the  contractor  succeeded  in 
collecting  all  data  needed  for  the  portfolio  analysis,  data  that  could  be  collected  in 
relation  to  the profile of the programme beneficiaries was  limited to  some action lines. 
Even  though  these  action  lines  accounted  for  about  80%  of  the  budget  committed,  it 
implies that the data related to the geographical and sectoral profile of other activities of 
the programme beneficiaries had to be estimated. 

Overall, the data collected through the seven targeted consultations can be considered as 
reliable; the survey population resulted representative for the overall population (or in the 
case  of  the  LGF  SME  beneficiaries,  sufficiently  representative  for  the  target  sample). 
There is one exception: in the case of the consultation targeting EEN SME clients, data 
on  the  profile  of  these  beneficiaries  was  not  available,  so  no  stratified  random  sample 
could be created (as was done for the LGF beneficiaries, instead). The distribution of the 
questionnaire to the SMEs by the intermediaries implied also that the evaluation had no 
control over the process; the lack in data on the profile of the EEN SME clients hindered 
a quality control of the representativeness of the survey population.  

Taking  this  uncertainty  on  the  reliability  of  the  data  into  account,  the  contractor 
complemented  and  triangulated  the  information  collected  through  this  survey  with  the 
(preliminary) data from the EEN Performance Enhancement System (PES) database and 
with the other monitoring data provided by EASME. 

The requirement to contact the SME beneficiaries of the larger actions (in the case of the 
EFG  even  for  the  ‘qualitative’  consultation)  only  by  using  a  cascade  approach,  i.e. 
through the intermediaries, deserves a specific note. The fact that the contractors' study 
team  did  not  have  direct  control  over  the  number  of  SME  clients  that  were  invited  to 
participate in the survey resulted problematic in some cases. In the case of the survey to 
the LFG beneficiaries, it implied that the contractor did not receive any response from the 
beneficiaries  located  in  France  because  the  financial  intermediary  did  not  circulate  the 
survey before the deadline, despite the  contractors' best  efforts.  In the case of the EFG 
qualitative  consultation,  only  three  out  of  five  intermediaries  were  available  for 
collaboration and the team received only four completed questionnaires (out of 12) from 
the EFG SMEs. In the case of the EEN clients' SMEs, the number of respondents from 
COSME-participating third countries was too low to make a meaningful analysis. 

The  contractor  therefore  endorsed  the  recommendation  made  by  the  CIP  and  EIP  final 
evaluation  teams  to  ensure  in  the  future  the  possibility  for  evaluation  teams  directly  to 
contact beneficiary SMEs. 

17 

 
As  regards  WATIFY,  the  main  weakness  of  the  methodology  lied  with  the  limited 
availability of data for the first phase of the campaign (2014-2015). Yet the monitoring 
data  collected  was  sufficient  to  formulate  answers  to  the  evaluation  questions  and  in 
some  cases,  present  comparisons  between  the  two  phases  of  the  campaign.  Therefore, 
adequate triangulation of qualitative and quantitative data collected both throughout the 
campaign’s  implementation  and  during  this  external  evaluation,  allows  for  the 
formulation of evidence-based answers to the evaluation questions. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

5.1.  RELEVANCE 

  How relevant is the COSME programme to the stakeholders (public and private) 

and the EU citizen? 

Both  intermediaries  and  beneficiary  SMEs  consider  the  financial  instruments  in 
COSME to be relevant to their needs, both in terms of the conditions set and the size of 
the financing offered - even though there are some issues remaining. 

The objective of the LGF is to help the financial intermediaries provide more loans and 
leases  to  riskier  SMEs,  thus  expanding  the  range  of  SMEs  they  can  finance.  The  ‘on 
demand’ principle and flexibility in the design of the programme aim at facilitating the 
adaptation  to  the  market  needs  and  to  the  needs  of  financial  intermediaries.26  The 
programme  supports  all  kinds  of  SMEs  that  face  difficulties  in  accessing  debt  finance, 
without  considering  any  specific  sector  or  any  specific  country  base,  but  rather  the 
SMEs’ level of risk for the lending sector. COSME takes a flexible approach and allows 
any institution implementing the LGF to increase its activity with ‘high risk’ or ‘higher 
risk’ SMEs  as per its  own definition of risk.27This greatly  facilitates the take up of the 
LGF.  

Nevertheless,  financial  intermediaries  mentioned  that  some  needs  are  not  sufficiently 
supported by the LGF, in their responses to the targeted consultation and/or interviews28. 
The  subordinated  loans  for  more  than  EUR  150k  to  an  innovative  SME  cannot  be 
supported  under  InnovFin  nor  under  COSME  and  the  loans  over  EUR  1.5  million 
(assuming a 50% guarantee rate). The surveyed LGF intermediaries also considered the 
time limitations to the guarantees (a minimum of 12 months and maximum of five years 
for  guarantee  amounts  above  EUR  1.5  million  and  10  years  for  those  below)  to  be 
inadequate for the needs of financial intermediaries and SMEs.  

The objective of the EFG is to crowd-in more capital for pan-European operations at the 
growth  and  expansion  stage.  Interviewed  stakeholders  regarded  this  focus  of  the  EFG 
instrument as appropriate. The flexibility to crowd-in the pan European Fund(s)-of-Funds 
is also regarded as relevant. However, the clause that the EFG can invest in a VC (only) 
up to 25% of the total amount is viewed by funds and the EIF as suboptimal for creating 

26 Co-financing from various EU programmes is also possible. 

27 There is no standard definition of “high risk” or “higher risk” SMEs among European financial intermediaries 

28  Pp  34  and  35,  Interim  Evaluation  of  the  COSME  Programme,  final  report,  Technopolis,  December  2017, 

https://ec.europa.eu/docsroom/documents/28084 

18 

 
 
                                                           
an attractiveness effect for other private investors. In comparison, EFSI can invest up to 
50%. 

The single web portal on EU finance (to be accessed through Your Business Europe or 
www.accesstofinance.eu)  addresses  the  information  asymmetry  failure.  It  provides 
contact  details  of  around  1,500  intermediaries  and  information  on  all  financial 
instruments under the new EU programmes 2014-2020 (COSME, EaSI, InnovFin under 
Horizon  2020,  Life  and  the  EU  Investment  Plan  under  EFSI,  and  the  new  financial 
instruments  supported  by  European  Structural  and  Investment  Funds).  It  also  provides 
information on how to access financial instruments from 12 different EU programmes in 
39  countries,  and  on  EU  financial  instruments  with  a  clean-tech  investment  focus  for 
climate-change objectives. The portal reached 150,000 monthly page views in 2015. 

According to the surveyed SMEs that benefited from the the EEN services, the network 
addressed three major failures29: 

  The broad range of information services, covering a.o. H2020 funding opportunities, 
information on financing opportunities and opportunities to increase competitiveness 
and innovativeness in the Single Market and beyond, address the clear (and ongoing) 
information asymmetry failure SMEs face to enhance their competitiveness 

  Through its advisory services, the EEN also addresses the capability failure, i.e. the 
SMEs’  need  for  know-how  on  management  processes,  customer  needs  and  how  to 
enter foreign markets  

  The  Network  also  addresses  networking  failures  by  helping  SMEs  to  find 
(international)  cooperation  partners  and  by  providing  an  avenue  for  intermediary 
organisations to collaborate.  

Confirmation  of  the  relevance  of  the  EEN  services  to  SMEs  can  be  found  in  the 
importance  attributed  to  these  actions  by  the  SME  clients  (Figure  1).  In  general,  the 
SMEs’  view  on  the  extent  to  which  Network  services  address  their  needs  is  quite 
positive. 

Figure 1: SME clients’ views on the importance of the EEN services 

29  P.  35,  Interim  Evaluation  of 

the  COSME  Programme,  final  report,  Technopolis,  December  2017, 

https://ec.europa.eu/docsroom/documents/28084 

19 

 
                                                           
Source: Technopolis, December 2017, https://ec.europa.eu/docsroom/documents/28084; p.36 

The tourism actions addressed the information asymmetry and networking failures. More 
than  60%  of  survey  respondents  thought  the  programme  largely  covered  these  failures. 
While the COSME actions therefore show a particularly high level of relevance for the 
SMEs  in  the  Tourism  sector,  the  responses  also  show  two  major  gap  areas:  the 
exploitation  of  synergies  between  the  tourism  offer  and  other  industries,  and  the 
enhancement of skills among tourism providers and their employees.  

In  the  field  of  Entrepreneurship,  the  COSME  programme  includes  information  and 
awareness-raising  measures,  several  portals  and  support  services,  networks  to  share 
knowledge  and  best  practices,  strategic  intelligence  platforms  and  measures  to  develop 
entrepreneurial  human  capital.  Besides  the  Erasmus  for  Young  Entrepreneurs  (EYE) 
programme,  COSME  also  funded  actions  on  digital  entrepreneurship,  women’s 
entrepreneurship, migrant entrepreneurship, senior entrepreneurship and entrepreneurship 
education, and early warning/ second starters.  

EYE has a high level of relevance to the needs of the respective target groups, with no 
major  gaps  identified.  A  large  numbers  of  Host  and  New  Entrepreneurs  who  had 
participated in the EYE programme during the 2014-2016 period were surveyed. These 
(potential)  entrepreneurs  assessed  the  extent  to  which  a  series  of  barriers  to 
entrepreneurship were present in their country today. Their responses provide widespread 
confirmation  of  the  existence  of  market  and  systemic  failures  in  this  area.  The  large 
majority  (jointly,  92%+)  agreed  that  entrepreneurs  face  obstacles  to  setting  up 
businesses, that new businesses struggle to survive and grow, and that there is a lack of 
effective policy support related to entrepreneurship. There was also a widespread (though 
less  strongly  held)  belief  that  there  is  a  lack  of  entrepreneurial  culture,  weak 
entrepreneurial spirit, and low levels of entrepreneurial firm creation in Member States.  

Figure below shows that more than 90% of these survey respondents agreed that COSME 
– through its largest entrepreneurship action, the EYE programme - helped to address all 
these areas to some extent, and particularly the lack of entrepreneurial culture and spirit.  

20 

31%42%43%46%46%54%56%59%43%34%42%36%36%36%33%32%26%25%15%18%17%10%11%8%0%10%20%30%40%50%60%70%80%90%100%FeedbackEncouraging the participation of SMEs in Horizon 2020"No wrong door" principleInnovation and transfer of technology and knowledgeInternationalisation beyond the EUAdvisory ServicesBusiness co-operationInformationImportance of the EEN services  / SME clientsn=1178-1199Very importantSomewhat importantNot at all important 
 
 
 
Source:  Technopolis,  December  2017,  https://ec.europa.eu/docsroom/documents/28084;  p.38In 
the  field  of  Internationalisation,  the  emerging  markets  are  covered  in  COSME  through 
the  clusters  internationalisation  and  IPR  Helpdesks.  A  survey  undertaken  by  the  ECCP 
platform  with  the  ESCPs’  project  coordinators  shows  that,  by  June  2016,  in  the  early 
phases of the implementation of the partnerships, 47.4% of them had selected USA as a 
target  for  international  cooperation.  Further  popular  markets  targeted  included  Brazil, 
Canada, China, Japan and Mexico.30 These are in line with the EU largest trade partners. 
Other  actions  such  as  the  EU’s  Low-Carbon  Business  Actions  (LCBAs)  in  Brazil  and 
Mexico, supported through the EU Foreign Partnership Instrument are examples of other 
approaches  targeting  the  matching  and  direct  funding  of  businesses  from  emerging 
European industries with partners from new emerging geographical markets31. 

The  2015  Flash  European  Barometer  on  Internationalisation  of  SMEs32  showed  that 
SMEs  considered  the  complicated  administrative  procedures,  high  delivery  costs,  and 
to  be  key  bottlenecks  for 
difficulties 
internationalisation.33 A fourth area of concern is the perceived high investment cost. 

identifying  business  partners  abroad 

in 

COSME  actions  such  as  the  EU-Japan  Centre34  and  the  IPR  Helpdesks35  are  very 
relevant in reducing the administrative burden. Actions providing support to identifying 
business partners, such as the European Strategic Cluster Partnership (ESCP) – Clusters 

30 ECCP, 2016: Analysis of the Consultation launched by ECCP amongst the ESCP-4i partnerships in June 2016,  

31 http://www.lowcarbonbrazil.com/index, http://www.lowcarbon.mx/  

32 

http://ec.europa.eu/COMMFrontOffice/publicopinion/index.cfm/Survey/getSurveyDetail/instruments/FLASH/sur
veyKy/2090 

33 European Commission, 2015: Flash European Barometer, Internationalisation of SMEs 

34 https://www.eu-japan.eu/ 

35 http://www.ipr-hub.eu/ 

21 

 
 
 
                                                           
Go  International  programme  (further:  ‘Cluster  Internationalisation’)  and  the  business 
missions  organised  in  the  context  of  the  European  Cluster  Collaboration  Platform 
(ECCP)36  and  the  EU-Japan  centre  are  very  relevant  to  the  overall  need  of  SMEs  in 
building trust in finding suitable partners. The COSME actions do not address the fourth 
barrier, which is the high costs that come with building trust in finding suitable partners. 
A  potential  upgrade  of  the  programme  could  therefore  include  other  types  of  support, 
such as demonstration projects, feasibility studies and knowledge-acquisition activities.  

For  the  EU  citizen,  the  relevance  of  the  programme  is  mainly  related  to  economic 
growth.  The  major  drivers  for  economic  growth  constitute  the  key  focus  of  COSME. 
COSME shows limited relevance for topics beyond the economic sphere, though, when 
considering  numbers  of  actions  funded  and  budget  allocated  to  them.  The  high-level 
objectives related to inclusive and sustainable growth are addressed mainly in an indirect 
manner. 

The  programme  relevance  related  to  sustainable  development  is  therefore  mainly  an 
indirect  one.  For  example,  among  the  Venture  Capital  (VC)  funds  that  received  EFG 
funding,  about  three  quarters  are  focusing  on  the  ICT  sector  and  clean  technologies. 
Societal  goals  are  also  the  funding  purpose  of  many  national  promotional  banks  that 
signed  an  agreement  for  the  LGF.  Sustainable  growth  was  explicitly  pursued  also 
through  the  Network  focus  on  ‘Measures  to  increase  SME  access  to  energy  efficiency, 
climate and environmental expertise’. 

  To  what  extent  are  the  initial  objectives  of  the  COSME  programme  (still) 

pertinent to the needs and problems it was designed to address? 

According to the supporting study, the four specific objectives are still relevant.  

First,  despite  the  economic  recovery  in  the  EU,  access  to  finance  remains  an  issue  for 
SMEs;  the  situation  remains  uneven  among  countries.  In  addition,  the  negative 
correlation  between  finance  and  firm  size,  age,  and  degree  of  innovation  that  emerged 
from the 2014 Survey on the Access to Finance of Enterprises (SAFE) seems to persist. 
The  2017  ECB  survey,  for  example,  shows  that  in  the  euro  area  large  enterprises 
continue to experience easier access to finance than SMEs, except in the case of equity 
financing.37  

The  explicit  objective  of  the  LGF  is  to  help  the  financial  intermediaries  in  providing 
more loans and leases to riskier SMEs that have difficulties in obtaining funding through 
the traditional banking system. LGF directly addresses the financing gap; the limits set to 
the tickets also ensure that the facility is of specific interest to SMEs. The objective of the 
EFG, instead, is to cover the needs of start-ups and SMEs in general in their growth and 
expansion stage.  

Second,  a  2015  EC  Staff  Working  Document38  indicated  that  in  the  field  of 
internationalisation,  performance  among  EU  SMEs  shows  limited  progress.  While  the 

36 https://www.clustercollaboration.eu/ 

37 Kraemer-Eis, H et al. (2016), The European venture-capital landscape: an EIF perspective, Volume I: The impact of 
EIF on the VC ecosystem, EIF Research and Market Analysis, Working Paper 2016/34. 

38 European Commission, (2015): Staff Working Document SWD(2015) 203 final on 28 October 2015 accompanying 
the Communication Upgrading the Single Market: more opportunities for people and business, COM(2015) 550.  

22 

 
                                                           
percentage  of  EU-based  SMEs  with  extra-EU  imports  and  exports  of  goods  improved 
since  2008,  the  European  Commission  reported  in  2014  that  only  25%  of  EU-based 
SMEs  were  engaged  in  export  activities  and  an  even  smaller  share  of  SMEs  (7%) 
exported beyond the EU. 

Third, reflecting the considerations made in  the  EC  Staff Working Document, both  the 
201539  and  the  2016  Annual  Growth  Surveys  (AGS)40  show  that  pursuing  structural 
reforms in the MS is key to modernising EU economies and ensuring better framework 
conditions  for  business.  Further  removing  regulatory  barriers  to  the  Single  Market, 
establishing the Digital Single Market and continuing the REFIT process for simplifying 
existing  legislation  have  been  highlighted  as  priorities  for  2015;  according  to  the  2016 
AGS,  increasing  transparency,  efficiency  and  accountability  in  public  procurement, 
ensuring  open  and  competitive  product  service  markets,  and  modernising  the  public 
administration remain further challenges in the EU.  

Fourth, fostering an entrepreneurial spirit among the European citizen has been a major 
policy objective during the last two decades. The SBA factsheet 2016 informs that at the 
EU28 level, there has been continuous (yet  moderate) progress  since 2008. Early stage 
entrepreneurial  activity  and  entrepreneurship  intentions  have  steadily  increased  since 
2008;  however,  the  factsheet  emphasises,  “more  policy  efforts  are  needed  to  turn 
entrepreneurship into a desirable career choice and to promote education as a means to 
develop an entrepreneurial attitude”. 

  How has the programme adapted to the political priorities, for instance to the new 
single market strategy and its particular objective to support start-ups and scaling-
ups?  

The policy context  of the COSME programme starts  with  the Europe 2020. The policy 
basis  of  the  COSME  programme  is  the  SBA  Review  of  2011.  The  Entrepreneurship 
Action  Plan  that  was  published  in  the  same  year  (2011),  as  well  as  the  Single  Market 
Strategy  and  the  Start-up  and  Scale-up  Initiative,  both  launched  in  2016,  constitute  the 
other building blocks for the strategic decision-making on the COSME Programme.  

The  Single  Market  Strategy  sets  the  need  to  support  SMEs  in  the  context  of  the  many 
obstacles  that  SMEs,  start-ups  and  young  entrepreneurs  are  facing,  such  as  access  to 
finance, the complexity of VAT regulations, aspects of company law and how to comply 
with  various  regulatory  requirements  in  different  markets.  The  Start-up  and  Scale-up 
Initiative was set up in  November 2016 to “reduce barriers for  young businesses in the 
Single  Market”.  It  followed  shortly  upon  the  launch  of  the  Single  Market  Strategy  and 
aligns  with  the  COSME  objectives,  namely  providing  SMEs  with  a  better  access  to 
finance  and  a  better  business  environment.  It  echoes  the  continuous  work  of  the 
Commission to foster better regulation41 (through actions such as REFIT and the High-

39 European Commission, (2016): Annual Growth Survey for 2015. Strengthening the recovery and fostering 
convergence  

40 European Commission, (2017): Annual Growth Survey for 2016. Strengthening the recovery and fostering 
convergence  

41 https://ec.europa.eu/info/law/law-making-process/better-regulation-why-and-how_en  

23 

 
 
                                                           
Level Group on Administrative Burden). The European Commission Investment Plan for 
Europe approved in 2014, also contributes to the COSME policy context.  

The portfolio of actions  in the COSME programme has achieved a high level of policy 
relevance. COSME responds particularly well to the priorities set out in the recent Single 
Market Strategy and Start-up and Scale-up Initiative (2016), setting a pronounced focus 
on  support  to  start-ups,  scale-ups,  and  second  starters,  but  includes  also  several  of  the 
priorities  defined  by  the  Juncker  Commission.  The  Entrepreneurship  Action  Plan  2020 
guides the design and implementation of the COSME entrepreneurship actions. 

The  flexibility  in  the  COSME  programme  allows  it  to  respond  in  a  swift  manner  to 
emerging  high-priority  topics  by  launching  new  actions  addressing  new  or  more 
emphasised  priorities,  and  by  steering  more  budget  towards  areas  that  constitute  a 
priority area of rising importance.  

The  WATIFY  campaign  is  underpinned  by  several  key  initiatives.  The  holistic  Europe 
2020 Strategy includes A Digital Agenda for Europe as one of its pillars. The objective 
of  the  Digital  Agenda  for  Europe  is  to  accelerate  innovation,  economic  growth  and 
quality of life by maximising the social and economic potential of ICT usage. Therefore, 
the  Digital  Single  Market  (DSM)  strategy,  launched  in  2015,  provides  businesses  and 
individuals  with  access  to  online  activities  without  restrictions.  The  DSM  specifically 
aims to: 1) provide better access for consumers and businesses in reaching online goods 
across Europe; 2) create favourable conditions for digital networks and services; and 3) 
maximise  the  growth  potential  of  the  European  Digital  Economy.  The  DSM  strategy 
encompasses  the  Digitising  European  Industry  initiative,  which  consists  of  five  pillars 
and provides a set of measures to help industry, SMEs, researchers and public authorities 
to maximise their gains from digitalisation as well as to support and link up national level 
actions to apply a coherent integrated approach to digitisation across the EU. 

Overall, the WATIFY campaign helps to put into action EU policy objectives in the field 
of industry digitisation and thus has strong relevance against EU policy context. 

5.2.  EFFECTIVENESS 

  To  what  extent  is  the  COSME  Programme  progressing  towards  its  specific 

objectives? 

The  financial  instruments  have  been  effective  in  reaching  their  outputs,  even  though 
progress  is  modest  for  the  EFG.  The  LGF  reached  a  considerable  number  of  SMEs, 
proposed  overall  more  favourable  financing  terms,  demonstrated  leverage  in  SME 
financing, and increased the supply of debt finance in most Member States. 

According the mid to long term objectives set in the annex of the COSME regulation for 
the  LGF  were  to  get  by  2020  a  Value  of  financing  mobilised  ranging  from  EUR  14,3 
billion  to  EUR  21,5    billion  and  a  number  of  firms  receiving  financing  which  benefit 
from guarantees from the COSME programme ranging from 220 000 to 330 000. 

The  European  Investment  Fund  (EIF)  has  –  in  the  time-period  evaluated  –  managed  to 
assign around EUR 611.7m (including the EFSI enhancement) allowing for a volume of 
EUR  5.5bn  in  debt  financing  provided  to  140,000  SMEs  for  the  LGF.  It  signed  EFG 
agreements  with  nine  financial  intermediaries  contributing  a  total  of  EUR  101m  to  the 

24 

 
 
selected  funds,  which  in  turn  provided  12  SMEs  with  risk  capital  to  support  their 
expansion  and  growth.  The  EFG  is  hereby  falling  behind  on  its  targeted  commitments. 
Geographically,  the  beneficiary  SMEs  are  concentrated  in  the  more  mature  financial 
markets. 

The  Enterprise  Europe  Network  (EEN)  has  shown  a  strong  capacity  to  reach  SMEs 
throughout  Europe,  including  150,000  SMEs  that  participated  in  the  EEN  co-operation 
and advisory services. Thanks to the EEN services, over 8,000 SMEs set up co-operation 
agreements42 with SMEs ‘across the border. The EEN services mainly supported SMEs in 
improving  the  quality  of  their  goods  and  services,  leading  to  (expected)  growth  in 
turnover and especially, entry to new (geographical) markets, both in Europe and beyond. 

The  Impact  Evaluation  of  the  Network  (2008-2014)43  concluded  that  the  average  net 
effect  for  SMEs  participating  in  the  Network  was  3.1%  increased  annual  growth.  This 
result  is  confirmed  in  the  COSME  interim  evaluation  where  SMEs  present  an  overall 
growth  rate  that  is  nearly  6.5  percentage  points  higher  than  a  comparison  group. 
Surveyed SMEs argued that the core role of the Network is to help companies grow and 
find  new  partners,  and  that  the  Network  services  are  adequate  in  doing  so,  which  is 
further  reflected  in  the  high  rates  of  satisfaction  where  86%  of  SME  clients44  surveyed 
are  satisfied  with  the  Network’s  services  and  93%  would  recommend  its  services  to 
others. 

The evaluation of the IPR SME helpdesks45 reveals that the action is very relevant  and 
that results indicate it could reach its objective on the medium and long term. Evidence 
shows  that  it  has  been  well  designed  to  make  EU  SMEs  aware  about  their  Intellectual 
Property Rights (IPR) in third countries' markets and contributes to the overall objective 
of the COSME programme. In  April 2017 the  IPR SME helpdesks had registered 3154 
helpline users. With 1347 registered helpline users against 1000 targeted, China performs 
well.  The  Latin-America  also  with  1217  registered  against  800  targeted.  Due  to  the 
belated implementation of its helpline, the South-East HD reached only 590 users against 
1000 targeted. 

The EU-Japan Centre for industrial cooperation was effective in fostering industrial and 
trade cooperation between the EU and Japan. Its  objective is to facilitate and foster the 
cooperation between European and Japanese companies and clusters by providing policy 
analysis,  services  to  businesses,  SME  support  (e.g.  on  tax  and  public  procurement), 
organising 
information  events  and  knowledge-sharing  opportunities  as  well  as 
networking activities.  

Activities organised by the EU-Japan Centre in 2014-2016 

Activity 
Policy seminars and analysis events 

Nr of activities / stakeholders reached 
In 2014, eleven events; 2015 ten events 
were held; in total, close to 3,000 

42 The long-term objective set by the COSME regulation is to have 2500 Partnership agreements signed per year. 

43https://publications.europa.eu/en/publication-detail/-/publication/d4cf03ed-972c-11e5-983e-
01aa75ed71a1/language-en 

44  The  long-term  objective  set  by  the  COSME  regulation  is  "Client  satisfaction  rate  (%  SMEs  stating  satisfaction, 
added-value of specific service): > 82 %" 

45  Interim  Evaluation  of  the  International  Intellectual  Property  Rights  (IPR)  SME  Helpdesk,  a  specific 

COSME measure, Technopolis, Annex 4 of this report. 

25 

 
                                                           
Company visits to address the practical adaptation of Japanese 
management and production systems to Europe 

participants  
Around five company visits per year, lasting 
one day each 

Webinars to inform EU SMEs of possible partnering opportunities 
and support for improving EU manufacturers’ competitiveness 
Cluster matchmaking events to inform EU SMEs of Japanese 
clusters and the opportunities for cluster-to-cluster partnerships 
Source: EU-Japan Centre for Industrial Cooperation. Activity Report for Fiscal Year 2015 

Since 2014, three actions per year, 
attracting between fifteen to twenty SME 

Average nr of participants per webinar: 107  

Under the framework conditions and competitiveness objective, most of the actions were 
still  under  implementation  at  the  time  of  the  evaluation.    Nevertheless  several  actions 
such as the SME envoy  Network, the Clusters Go international or the Tourism actions, 
have  reached  the  milestones  in  terms  of  outputs  set  in  the  COSME  yearly  work 
programmes.  

Overall, stakeholders are satisfied with the results of the SME policy actions. The Envoys 
perceive the network as a highly valuable EU-level forum where ideas and practices are 
exchanged.  The  SME  Performance  Review  as  well  as  the  factsheets  are  used  by  the 
national  representatives  and  are  considered  to  be  of  high  quality.  Many  events  are 
organised  throughout  Europe  during  the  SME  Week,  and  the  SME  Assembly  gathers 
stakeholders  from  all  MS.  The  main  outreach  tool,  Business  Planet,  is  broadcasted  on 
Euronews.  There  is  a  general  perception  that  the  activities  organised  at  the  EU-level 
contribute to a convergence of SME policies across COSME countries. 

For the Tourism actions, the consulted stakeholders and beneficiaries indicated as direct 
results  the  creation  of  new  strategic  partnerships  across  borders  and  more  visibility  in 
international markets of touristic destinations in the EU. The Tourism actions are geared 
towards  the  involvement  of  actors  along  the  value  chain.  Based  on  the  data  available, 
they  directly  involved  about  100  companies,  active  in  the  tourism  sector,  50  private 
enterprises  active  in  other  sectors  and  200  organisations  (  mainly  business  associations 
active in tourism), public agencies, user organisations, and professional service providers 
(e.g. marketing specialists). 

The  "Clusters  Go  International"  actions  show  good  potential  to  lead  to  an  improved 
strategic  position  in  global  value  chains  and  an  enhanced  access  to  potential  inward 
investors as survey respondents and interviewees alike highlighted that the activities are 
based  on  a  strategic  analysis.  The  limited  budget  is  flagged  as  a  hurdle  for  their 
internationalisation  activities  and  was  reported  by  90%  of  the  clusters  as  an  important 
bottleneck. 

In  October  2017,  the  Clusters  Go  International  action  involved  25  European  Strategic 
Cluster  Partnerships  (ESCPs),  launched  under  the  "Clusters  Go  International  action 
representing  145  cluster  organisations  across  23  European  countries  and  more  than 
17,000  European  SMEs.  Only  15  out  of  the  25  selected  ESCPs  were  funded  due  to 
budget  constraints.  They  account  for  approximately  95  clusters  reaching  out  to  over 
10,300 European SMEs. The Clusters Go  International  action  succeeded in  involving a 
high number of clusters, some even at a voluntary level, testifying the high interest in this 
action in  the EU  cluster  community. Moreover, projects  show  overall  good progress  in 
reaching the outputs expected, such as partnership agreements.  

The overall achievements of the 1st generation of European Strategic Cluster Partnerships 
(ESCPs)  for  going  international  2016-2017  prove  their  success  in  supporting  SME 
internationalisation.  Over  a  24  months  period,  they  involved  2000  SMEs  in  activities 
targeting  international  third-markets  generating  85  concrete  business  cooperation  cases 
with international partners. 370 Cluster-to-Cluster events and 3010 Business-to-Business 
26 

 
events  have  been  conducted,  whilst  39  Memoranda  of  Understanding  and  45 
collaboration  projects  implemented  between  EU  clusters  and  international  peer 
organisations.  The  interest  in  the  action  remained  high  for  2018-2019  with  23  new 
partnerships established since46. 

The e-skills action reached the outputs expected. Stakeholders interviewed were positive. 
In  2016,  it  covered  21  different  countries  where  events  were  organised  and  many 
stakeholders  and  high-level  policy-makers  were  involved,  including  members  of  the 
European  Parliament.  Stakeholders  and  survey  respondents  unanimously  indicated  the 
training 
effective  engagement  of  various  stakeholders  such  as  education  and 
organisations,  employees  and  entrepreneurs  as  the  most  important  factor  in  driving 
progress  towards  an  enhanced  awareness  and  especially  an  enhanced  availability  of 
digital skills. 

The  two  WATIFY  campaigns  managed  to  achieve  4  of  the  5  specific  objectives:  to 
promote  the  transformation  of  industry  through  advanced  technologies  by  sharing 
success stories ; to stimulate digital entrepreneurial culture by identifying and promoting 
digital transformation of SMEs and digital start-ups and their success (or failure) stories 
at  pan-European, national,  regional and local  levels; to  spark and support new business 
ideas  in  the  digital  economy  and  boost  their  successful  commercialisation;  and  to 
strengthen  business  confidence  in  the  application  and  added  value  of  digital  and  key 
enabling  technologies,  business  processes  and  new  business  models.    However, 
WATIFY  2  did  not  succeed  in  supporting  Europe’s  regions  to  translate  their  smart 
specialisation strategies in the area of digital and key enabling technologies. Neither desk 
research, nor interviews  were fruitful in  collecting information  on the support provided 
by  WATIFY  to  regions  for  the  translation  of  their  smart  specialisation  strategies  into 
actual projects. 

Under the Entrepreneurship objective, the Erasmus for Young Entrepreneurs (EYE) was 
successful  in  reaching  its  expected  outputs  and  outcomes.  It  involved  in  total  around 
2,000 New Entrepreneurs and we estimate that after approximately a  year and a half, it 
led  to  the  creation  of  250  new  companies,  EUR  5  million    of  additional  turnover,  and 
about 1,000 new jobs. The most prominent weakness of this action line lies in the small 
scale of its activities. 

  What  are  the  factors  driving  or  hindering  progress  and  how  do  they  link  with 

different actions of the programme? 

The key factor of success for COSME is to rely on an extended network of intermediaries 
that ensures proximity to SMEs and to stakeholders.  This is particularly the case for the 
EEN, the clusters or EYE but also for the financial instruments.  

The  LGF  has  reached  a  high  level  of  take-up  thanks  to  the  flexibility  of  its  design. 
However,  financial  intermediaries  report  high  administrative  costs  for  entering  into  an 
agreement  with  the  EIF  and  complying  with  its  requirements  are  problematic.  The 
threshold  of  EUR  150  000  above  which  the  financial  intermediaries  must  check  if  the 
SME does not meet any of the innovation criteria for funding under the H2020 InnovFin 
instrument  creates  the  negative  effect  of  inducing  the  intermediaries  to  limit  their 
financial offer to  this  threshold. Most important,  the threshold is  deemed  inadequate to 

46 https://www.clustercollaboration.eu/eu-cluster-partnerships/escp-4i/first-generation/achievements and 
https://www.clustercollaboration.eu/news/clusters-go-international-partnering-event-2018-sharing-experiences-boost 

27 

 
                                                           
the needs of many SMEs. The EFG is  falling behind on its targeted commitments. The 
growth projects of these SMEs mainly focus on expansion in the existing (local) markets 
and  the  development  of  new  products  and  services.  The  key  barrier  identified  in  the 
supporting study is the maturity of Venture Capital markets in the EU. Geographically, 
the beneficiary SMEs are so far concentrated in the more mature financial markets. 

The  effectiveness  of  the  EEN  services  relate  to  the  profile  of  the  EEN  member 
organisations and their embeddedness in their national and regional environments. There 
is a close relationship between the services provided in the context of COSME and these 
organisations’  ‘usual’  services.  The  credibility  of  the  organisation  in  delivering  quality 
services  is  a  primary  condition  for  the  use  of  the  EEN  services  by  SMEs.  Another 
enabling  factor  is  the  proximity  of  the  members  to  the  local  SMEs,  as  well  as  the 
potential for knowledge sharing and information.  However, SMEs are not always aware 
that  the  EU  funds  EEN  services  and  SMEs  awareness  of  the  existence  of  the  Network 
could be improved. 

Despite  the  success  of  the  Erasmus  for  Young  Entrepreneurs,  the  entrepreneurship 
objective does not seem to have a strategic focus in COSME work programmes. Actions 
originate  from  different  places  and  are  then  managed  across  multiple  Commission 
services, with only informal and unstructured interaction across them. 

Except  for the financial  instruments  and the EEN, the fragmentation of  the programme 
into many smaller activities for 20% of the programme reduces the potential  impact  of 
actions.  This is particularly the case for EYE and for the Clusters. 

  To  what  extent  is  the  Programme  contributing  to  the  five  union  priorities  for  a 

sustainable and inclusive growth? 

The COSME intervention logic indicates that contributions  by the COSME programme 
to  the  higher-level  EU  policy  priorities  are  expected  in  four  areas:  an  increase  in  ‘Jobs 
and  growth’,  an  enhanced  ‘Business  creation  and  growth’,  a  strengthened  ‘Global 
competitiveness of the SMEs’, and an increase in ‘Inclusive and Sustainable Growth’. 

In the graphs below we first set these data in the context of statistical data related to the 
SMEs’ employment shares at the sectoral level (Figure 2) and the developments from this 
perspective  in  recent  years  (Figure  3).  Sectors  with  the  highest  shares  of  COSME 
beneficiaries are highlighted in dark red; those with high shares are highlighted in pink. 

This  analysis  suggests  that  COSME  has  a  high  potential  for  contributing  to  the 
attainment of the Jobs and Growth objective in sectors that are the most important for the 
EU economy. 

However, as explained in the relevance section, its potential to reach impact in relation to 
global  and  societal  challenges  is  limited.  The  main  COSME  Instruments,  such  as  the 
Enterprise  Europe  Network  or  the  Loan  Guarantee  Facility  target  all  SMEs  without 
distinction  of  sector  or  activity.  This  universal  offer  means  that  the  support  to  policy 
objectives such as Climate change, Gender Mainstreaming or Youth Unemployment can 
only be indirect and due to undesired effects. 

Figure 2: SME employment shares at sector level 

28 

 
Source: Annual report on European SMEs 2015/2016; Eurostat, Statistical Offices & DIW Econ 

Figure 3: Change in employment in SMEs 

Source: Annual report on European SMEs 2015/2016; Eurostat, Statistical Offices & DIW Econ 

5.3.  EFFICIENCY 

  Are  the  costs  generated  by  the  programme  proportionate  to  the  benefits 

generated? 

 COSME  is  operating  at  a  reasonable  level  of  efficiency  overall,  with  its  principal 
components all being implemented in line with the specific objectives and timetables set 
out  in  the  annual  work  programmes.  The  feedback  from  beneficiaries  is  generally 
positive  with  some  exceptions.  A  substantial  majority  is  reporting  a  good  cost-benefit 
ratio  for  their  individual  participations.  For  example,  more  than  two  thirds  of  EEN 
respondents  (68%,  158  respondents)  reported  that  the  benefits  outweigh  the  time 
investment.  

Surveyed beneficiaries report some issues of administrative burden. For example, half of 
the  Cluster  Go  International  (CGI)  Participants  responding  to  the  survey  thought  the 
duration of the CGI project funding was not appropriate; about 40% of them thought the 
funding  needed  for  the  internationalisation  activities  was  not  correctly  estimated.  The 
financial  intermediaries  indicated  that  the  reporting  requirements  are  burdensome  and 
risk  reducing  the  attractiveness  of  the  LGF.  They  also  argued  that  the  EFG’s 

29 

0%5%10%15%20%25%30%Employment shares at sector level in 2015 (EU28) -SMEs-2%-1%0%1%2%3%4%5%6%7%Change in employment 2013-15 (%) -SMEs 
 
 
administrative burden is substantially higher than is normal in the Venture Capital (VC) 
sector.  

After three years of experience under the COSME programme, the IPR SME helpdesks 
has been showing effectiveness and should now focus the resources to gain in efficiency. 
To  that  effect,  improved  service  quality  is  needed,  as  well  as  tackling  the  problems  of 
insufficient promotion of activities and improving partnering services. There is no need 
for  additional  means  in  order  to  achieve  more  impact,  however  the  use  of  the  current 
ones could be optimised. The fact that intermediaries act as multipliers, can be seen as an 
efficiency lever, provided the visibility increases concurrently thanks to more promotion 
efforts. The  strategy  to  reach  the  critical  mass  of  SMEs  needs  to  be  improved.  The 
evaluation  concludes  on  the  need  for  continued  funding  for  the  three  regions  and 
considers extending this type of service to India. 

The two WATIFY campaigns cost EUR 1,6 million and allowed to reach some 200,000 
participants through an initial and closing event, matchmaking events, awareness raising 
campaign, video clips and case studies.  Participants to the matchmaking events insisted 
on their usefulness and even asked for more of these events.  On the awareness  raising 
events there could have been less for the same effect and usefulness. For the launching 
and  closing  events,  the  co-organisers  mentioned  that  it  generated  some  administrative 
burden.  Surveyed  SMEs  agreed  that  their  participation  has  resulted  or  will  result  in  a 
number  of  benefits  for  their  company:  their  increased  competitiveness  in  the  market, 
higher  profit  margin,  higher  quality  of  their  products,  more  clients,  new  business 
partnerships, support in introducing technological changes and funding opportunities for 
technological  changes.    Overall  beneficiaries  found  the  campaigns  useful  and  cost-
effective.  Several  potential  improvements  were  identified:  The  delayed  launch  of  the 
WATIFY  website  prior  to  the  launch  of  the  second  campaign  created  obstacles  for  the 
promotional  activities;  The  timing  in  organizing  the  events  caused  some  inefficiencies 
and unexploited potential; The significant Facebook presence in countries outside the EU 
Member  States  or  COSME  programme  countries  for  the  second  phase  of  WATIFY 
shows that this communication channel could have been used in a more efficient way.   

  What  are  the  regulatory/administrative  costs  and  the  benefits  for  the  different 

stakeholders? 

COSME’s  contribution  to  EASME  for  the  period  2014-2016  was  EUR  21,7m,  which 
covered the operational (administrative) aspects of the programme. This is approximately 
EUR 7,2 million per year and represents 6.6% of the budget committed to COSME in the 
same period, excluding the budget for ‘Access to Finance’ managed by EIF.47  

For the EEN, approximately 600 Network member organisations are grouped in consortia 
organised at the national or regional level that have been selected through open calls. On 
average,  approximately  5.5%  of  the  COSME  budget  made  available  to  the  Network 
member organisations is dedicated to management/coordination activities.  

47 General budget of the European Union for the financial year 2018 (http://eur-
lex.europa.eu/budget/data/DB/2018/en/SEC03.pdf), 2017 http://eur-lex.europa.eu/budget/data/DB/2017/en/SEC03.pdf)  
and 2016 (http://eur-lex.europa.eu/budget/data/DB/2016/en/SEC03.pdf). Based on outturn figures recorded in those 
documents (i.e. expenditure from previous years). 

30 

 
 
                                                           
The  EYE  European  Partnerships  (EPs)  are  consortia  of  Intermediary  Organisations 
officially awarded by EASME, following an open call for proposals. Their staff costs are 
included in the grant provided by COSME 

The  European  Investment  Fund  (EIF):  The  EIF  acts  as  entrusted  entity  for  the 
implementation  of  the  COSME  financial  instruments  through  a  mandate  given  by  the 
Commission. For the period 2014-2016, the costs for the COSME implementation by the 
EIF amounted in a total of EUR 4.5 million for EFG and EUR 17 million for the LGF. 
These figures corresponded for EFG to 2.6% of the total EU budget committed and for 
LGF to 4.5%.  

The supporting study48 presented several Cost Effectiveness Analysis in terms of number 
of SMEs reached, the increase in turnover or the employment generated.   

For example: 

Each EUR 1 million invested in the EEN has led to an increase in employment of circa 
377  people  among  client  SMEs.  This  is  based  on  an  increase  of  two  employees  per 
company  in  the  period  2014-2016  (calculated  from  our  SME  client  survey),  the  total 
number of companies supported (210,000), and the additionality factor. 

Each EUR 1 million invested in LGF has led to an increase in employment of circa 235 
people  among  client  SMEs.  The  evaluators  further  estimated  –  after  accounting  for  the 
additionality  –  that  for  each  EUR 1 million  invested  into  the  LGF  (effects  fully 
attributable to the Loan Guarantee Facility): 

  additional employment of 491 will be created; 
  additional EUR 22 million in turnover will occur in treated SMEs; 
  470 SMEs will be supported.  

Each EUR 1 million invested in EYE has led to an increase in employment of 33 people 
among  SMEs  (Hosts)  and  New  Entrepreneurs.  Again,  this  a  very  positive  result; 
however,  it  implies  that  EEN  and  LGF  are  more  cost-effective  than  EYE  in  terms  of 
supporting job creation. 

The Cluster Go International action reached 3,800 SMEs per EUR 1 million invested. 

  How efficient is the implementation structure and governance of the programme? 

The main strength of COSME is the clarity of its work programme descriptions, leading 
to  a relatively small share of ineligible proposals  responding to  the calls for grants.  No 
major  administrative  bottlenecks  were  identified  and  the  simplification  measures  that 
were  introduced  were  overall  considered  as  positive,  even  though  the  new  IT  system 
clearly had some teething problems.  

A  major  weakness  of  the  programme  design  is  the  unbalanced  distribution  and  strong 
fragmentation of the budget available. As stipulated by the COSME regulation, the large 

48  Interim  Evaluation  of  the  COSME  Programme,  final  report,  Technopolis,  December  2017, 

https://ec.europa.eu/docsroom/documents/28084 

31 

 
 
                                                           
majority of the COSME budget (80%) is used for the two key actions.49 The remaining 
20%  of  the  budget  is  spread  over  a  large  number  of  small  actions.  This  fragmentation 
negatively influences the potential for cost-efficiency in the programme implementation 
and  accentuates  the  limits  in  strategic  steering  and  coordination  of  the  programme. 
Competition in some COSME calls can be quite high, especially for the smaller actions, 
which can be considered a direct consequence of the funding fragmentation. This places 
an additional burden on applicants, which can be particularly problematic for SMEs and 
may lead to a narrowing of the applicant base, both geographically and sectorally.  

Regarding  the  contribution  of  EASME  to  an  efficient  management  of  the  COSME 
programme, two  issues  can  be  noted.  First,  the  time-to-grant  for  the  open  calls  was 
shorter than the target set by EASME. Second, the number of small actions delegated to 
the agencies requires the involvement of a proportionally higher number of staff than for 
bigger  actions  such  as  the  EEN.  In  addition,  surveyed  beneficiaries  indicated  also  the 
difficulties in using the EASME IT systems in the first years of the programme and the 
user-friendliness of the IT tools. 

Another major weakness of the programme is the quality of data management. The lack 
in quality and especially completeness of the data on the beneficiaries of the programme 
constitutes  a  major  hurdle  for  an  efficient  management  of 
the  programme 
implementation. The spread of the implementation responsibilities over the many actors 
involved  (the  EIF,  EASME  and  in  2014-16,  about  15  DG  GROW  units)  creates  a 
considerable challenge to efficient programme management and coordination, with a risk 
of  weakened  programme  oversight.  It  also  makes  it  very  much  harder  to  evaluate  the 
programme.  The  various  reporting  systems’  focus  on  activities  and  outputs  is  equally 
problematic.  A  stronger  attention  for  the  collection  of  data  on  the  profile  of  the 
beneficiaries is needed to allow for a proper monitoring of the programme outcomes and 
progress towards reaching its objectives. 

5.4.  COHERENCE 

  Are the different actions of COSME implemented coherently between them? 

Overall,  the  COSME  programme  shows  a  good  level  of  internal  coherence:  efforts  are 
increasingly  being  made  to  create  synergies  and  no  substantial  overlaps  could  be 
identified. The EEN services are the most-often mentioned targets for synergies. The risk 
is to overcharge the EEN with responsibilities to which the Network cannot respond in an 
efficient way, and with the risk of losing the focus on its core activities. More synergies 
could be created by taking a more integrated, cross-thematic approach. 

The  communication  activities  related  to  the  financial  instruments  created  a  certain 
abundance  of  information;  those  related  to  the  entrepreneurship  actions  appeared  to  be 
run in  isolation from  each other. The assessment is  positive in  relation to the synergies 
and  complementarities  created  between  the  Your  Europe  Business  portal  and  the  EEN 

49  Interim  Evaluation  of  the  COSME  Programme,  final  report,  Technopolis,  December  2017, 

https://ec.europa.eu/docsroom/documents/28084, p.112 

32 

 
 
                                                           
 
services,  and between other COSME activities  geared towards internationalisation such 
as the EEN and Clusters Go International programme. 

There is a positive coherence in relation to the complementarities and synergies created 
between  Clusters  Go  International  with  other  COSME  activities  geared  towards 
internationalisation and the emerging markets of the main EU trade partners are covered 
through the action50. 

A better cooperation between the IPR SME helpdesks and the Cluster Internationalisation 
actions  should  be  a  priority  for  the  future.  The  interaction  between  the  three  helpdesks 
should  be  maintained,  as  well  as  with  other  COSME  initiatives  (mainly  the  EEN)  and 
other EU and national initiatives. 

  How are the different COSME actions overlapping/synergizing with the activities 

of ESI funds, H2020, EFSI, EASI and other EU programmes? 

The division of tasks between COSME and H2020 is clear and we noted no significant 
overlaps between COSME and H2020 or other EU activities. Nevertheless, there is room 
for  the  creation  of  more  synergy  and  complementarities,  especially  for  the  SME 
internationalisation  activities.  Overlaps  between  the  COSME  LGF  and  financial 
instruments  established  under  ESIF  may  occur  when  addressing  similar  SMEs  in  the 
regions.  The  latter  was  also  highlighted  in  the  independent  Evaluation51  of  the  EFSI 
Regulation,  which  was  concluded  on  4  June  2018,  namely  that  there  is  a  risk  of 
competition between ESIF financial instruments and LGF, and even indirectly EFSI.  

The more than 625 members of the EEN help about 250.000 SMEs per year to increase 
their  competitiveness  and  innovativeness  in  the  Single  Market  and  beyond  by  offering 
integrated  services.  There  is  a  clear  market  failure  for  SMEs  regarding  information, 
funding, accessing new markets and networking to internationalize their business outside 
the home country.  Network organisations provide information  on funding opportunities 
made  available  by  the  various  EU  programmes  such  as  COSME  financial  instruments, 
Horizon2020 SME Instrument and Structural Funds. 

The  Cluster  Go  international  can  also  play  a  major  role  to  incentivise  European  SMEs 
across regional industrial ecosystems to enter into joint business projects with partners in 
third  countries52  for  other  related  actions,  such  as  the  EU’s  Low-Carbon  Business 
Actions  in  Brazil  and  Mexico,  supported  through  the  EU  Foreign  Partnership 
Instrument53. 

  To  what  extent  has  the  coherence  been  maximised  in  view  of  national 

programmes? 

50 The horizontal actions of the European Cluster Collaboration Platform and the European Observatory for Clusters 
and Industrial Change also provide support within the umbrella of cluster initiatives. 

51 https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-
activities/independent-evaluation-european-fund-strategic-investments-european-investment-advisory-hub-and-
european-investment-project-portal_en 

52  In  Mexico,  at  least  50%  of  European  participants  were  informed  about  the  LCBA  project  through  cluster 
organisations. 90% of those clusters were members of the European Cluster Collaboration Platform (ECCP) 

53 http://www.lowcarbonbrazil.com/index, http://www.lowcarbon.mx/ 

33 

 
                                                           
The  interplay  between  the  EU,  national  and  regional  levels  of  support  for  SMEs  is  not 
always  synergistic  and  complementary,  even  though  no  substantial  overlaps  were 
identified.  Especially  for  actions  where  proximity  to  the  SME  communities  is  a  strong 
enabling  factor,  better  coordination  with  national  and  regional  actions  would  further 
improve  coherence.  This  is  especially  the  case  for  the  EEN  services  and  the  COSME 
actions supporting cluster organisations. 

Both EFG and LGF have adopted a bottom up approach, have the flexibility to combine 
with  national  programmes  and  appear  to  be  rather  complementary  with  other  schemes. 
However, Surveyed LGF intermediaries see both complementarities (9 out of 16 intermediaries) 
and overlaps (10 out of 16) of the LGF instrument with national level actions. 

Coherence with initiatives at the national/regional levels is a topic of relevance especially 
for the EEN services. The Network builds on regional players that are embedded in the 
local environment. The survey with SME clients of the Network shows that most clients 
were  unsure  whether  the  Network  services  are  complementary  to,  or  overlap  with, 
existing national or regional support in their own country; the findings from the survey 
with  Network  members  showed  similar  results.  During  the  interviews,  the  Network 
intermediaries  explained  that  they  actively  seek  this  complementarity.  For  example, 
when prioritising their activities, they aim to focus only on those services that are not yet 
available in their region. Moreover, many intermediaries explained that they have a good 
collaboration with other  stakeholders that provide services  to SMEs in  the region; they 
organise  joint  events  or  activities,  and  actively  seek  to  coordinate  their  actions.  These 
national  and  regional  stakeholders  include  ministries,  municipalities,  chambers  of 
commerce, EU info centres, specialists in IPR, lawyers, etc. By means of signposting, the 
Network intermediaries refer SMEs to other stakeholders in the field and vice versa when 
relevant. 

In  conclusion,  while  just  over  half  of  the  Network  clients  and  members  feel  that  the 
Network  services  are  complementary  to  services  provided  at  national  or  regional  level, 
there is scope for improvement. 

In the context of the Cluster internationalisation programme, interviewees indicated that 
there is no clear coordination between EU-level and national level cluster policies, which 
is perceived as problematic especially in the context of the limited funding available for 
the cluster internationalisation actions in COSME. The interviewees considered that the 
limited COSME funding for CGI might be considered as an incentive for clusters to co-
invest also their own resources to achieve the desired results. They emphasised that this 
approach does not sufficiently take account of the diverging levels of funding for clusters 
in the EU Member States. According to these interviewees, this approach hinders a level 
playing  field  for  cluster  organisations  in  countries  without  direct  cluster  policies.  They 
therefore  emphasised 
improved  coherence  and  creation  of 
complementarity in the EU funding for cluster internationalisation with the availability of 
national  level  funding.  In  this  context,  it  is  to  be  noted  that  only  3%  of  the  CGI 
participating clusters are located in EU13 countries. 

the  need  for  an 

The  complementarity  of  WATIFY  with  similar  national  initiatives  emerges  from  its 
European  character.  Besides  these  initiatives  of  a  more  cross-cutting  nature,  there  are 
ones  focused  on  specific  topics  /  sectors.  Some  countries  that  used  WATIFY  materials 
agreed  that  while  there  are  similar  initiatives  to  WATIFY,  it  is  important  to  have  a 
European  initiative  encouraging  enterprises  to  digitalise.  Internally,  the  WATIFY 
campaign was overall coherent (in terms of there not being conflicting events or activities 
is  a  complementary  aspect  between  awareness  raising  and 
organised).  There 

34 

 
matchmaking  events.  The  latter  were  organised  under  WATIFY  umbrella,  and  often 
included  the  cooperation  with  local  or  regional  stakeholders.  The  former,  in  contrast, 
were  organised  by  various  actors  in  different  European  countries,  and  were  used  by 
WATIFY  as  platforms  to  introduce  and  promote  the  campaign,  its  objectives  and 
messages.  

5.5.  EU ADDED VALUE 

  What is the added-value of the programme and actions being conducted at the EU 

level? 

COSME has a good level of European added-value. The European dimension constitutes 
the very essence of the design of its actions and is crucial for their implementation and 
effectiveness.  In  most  cases,  the  ‘subsidiarity’  of  these  actions,  i.e.  the  benefit  of  their 
implementation at the European rather than national or regional levels, is therefore very 
high.  While  many  national  and  regional  initiatives  also  seek  to  strengthen  the 
competitiveness  of  SMEs,  the  scale  of  support  through  COSME  and  its  availability  to 
financial  and  business  support  intermediaries  in  every  EU  Member  State  are  quite 
distinct and highly additional. In several cases, the EU level actions have not only been 
additional  to,  but  have  also  helped  in  enhancing  national,  regional  and  local  level 
measures. 

Indeed, COSME provides important financial leverage that allows national and regional 
intermediaries to provide lending products, which have a higher risk profile compared to 
their  normal  product  offering,  and  it  allows  offering  support  to  a  substantially  bigger 
number of businesses than they might otherwise be able to service. Furthermore, national 
support programmes, especially in smaller markets, have higher leverage when combined 
with  an  EU-level  programme.  However,  the  added-value  of  the  LGF  suffers  from  the 
non-targeted  approach,  which  so  far  has  implied  that  the  support  did  not  sufficiently 
reach the less mature financial markets. In terms of EFG financing, intermediaries very 
clearly stated that EU action is additional, despite that as recognised by the independent 
Evaluation54  of  the  EFSI  Regulation,  which  was  concluded  on  4  June  2018,  there  were 
initial  overlaps  with  the  Expansion  and  Growth  Window  under  the  EFSI  Equity 
instrument.  Despite  that  the  two  instruments  largely  have  the  same  investment  focus, 
operations have been allocated to EFSI since it has greater resources and can offer more 
favourable conditions. 

Moreover, the analysis from the supporting study reveals that the financing – supported 
by the LGF for example – also has triggered an additional investment from 21 % to 29 % 
as shown in the figure below.  

54 https://ec.europa.eu/info/evaluation-reports-economic-and-financial-affairs-policies-and-spending-
activities/independent-evaluation-european-fund-strategic-investments-european-investment-advisory-hub-and-
european-investment-project-portal_en 

35 

 
 
 
 
                                                           
Source: COSME Interim Evaluation (Technopolis, based on survey data)   

Further,  by  taking  into  account  the  actual  value  of  the  principal  financing  received  by 
those SMEs, the evaluators have estimated that LGF has de facto triggered approximately 
EUR 1,25 billion as additional investments as shown in the figure below. 

Source: COSME Interim Evaluation (Technopolis, based on survey data)   

The  distribution  in  the  two  figures  above  is  based  on  the  SME  definition.  From  the 
graphical  presentation  in  both  figures,  it  is  evident  that  whereas  the  medium  or  large 
enterprises  trigger  higher  additional  investment  percentage  wise,  i.e.  29 %,  the  smaller, 
but more numerous, the enterprises are, the greater value of additional investments will 
flow in the EU economy, i.e. EUR  959 million. 

The  supporting  study  also  has  identified  via  a  dedicated  survey  that  65 %  of  the 
respondent SMEs are of the opinion that they would have borne negative consequences 
for their business plans in absence of LGF support.  

36 

 
 
 
 
 
 
Source: COSME Interim Evaluation (Technopolis, based on survey data)   

As shown in the figure above, these negative consequences for 65 % of the SMEs mean 
de  facto  that  44 % would  have  continued  their  business  project  on  a  smaller  scale  and 
21 % would not have been able to implement their business plans at all. 

The European dimension is  at the core of the Enterprise Europe Network.  It  allows for 
the  attainment  of  a  scale  and  quality  of  the  services,  and  in  some  cases,  even  the 
implementation of the service as such, that would otherwise not be possible. The EEN’s 
role  as  an  intermediary,  involving  Chambers  of  Commerce  and  national/regional 
innovation  agencies,  provides  the  advantage  of  a  gateway  for  reaching  SMEs;  its  EU 
added-value  could  be  strengthened  by  focusing  on  those  services  that  build  upon  the 
transnational characteristics of the Network itself, i.e. the internationalisation and single 
market penetration. Options to ensure an appropriate level of co-financing and/or funding 
from EU funds could be examined. 

The  IPR  SME  helpdesks  add  value  at  EU  level  by  developing  and  broadening  their 
expertise  to  all  EU  SMEs.  In  order  that  any  SME  in  any  EU  MS  have  access  to  an 
equivalent quality of IPR service, it is of specific value to be able to offer them an IPR 
one-stop-shop to accompany them in their globalisation projects. 

The Cluster Go International program (with its fostering of European partnerships among 
clusters in order jointly to develop and implement internationalisation strategies) shows 
very  high  EU  added  value,  where  the  scale  and  depth  of  support  goes  well  beyond  the 
support that can be offered to European SMEs through national or regional initiatives55.  

A  separate  evaluation  of  cluster  initiatives  from  201456  concluded  for  instance  that  the 
European  cluster 
through  structuring  cluster 
cooperation contributing to the emergence of European meta-clusters" and have reached 

initiatives  "generated  added  value 

55 The European cluster initiatives are designed to complement and support efforts taken at regional and national level, 
with a particular focus on supporting strategic interregional partnering. 

56  Technopolis,  EY  &  Danish  Technologica  Institute  (2014)  Evaluation  -  Cluster  initiatives  managed  by  DG 
Enterprise  and  Industry,  ENTR/172/PP/2012  –  LOT4,  available  at  https://publications.europa.eu/en/publication-
detail/-/publication/c1ad8410-9bae-49df-b1f1-69b36071cf30/language-en 

37 

 
 
 
 
                                                           
 
out to a wide range of clusters, while flagging out that they lacked the financial means to 
involve SME sufficiently. 

  To  what  extent  and  why  do  the  issues  addressed  by  the  programme  continue  to 

require action at EU level? 

The  enhanced  added-value  for  the  Union  of  the  proposed  financial  instruments  lies  in 
addressing market failures that cannot be addressed by Member States.57 Funding at the 
supranational level in Europe allows the continent to match levels of funding seen in the 
United States. A discontinuation of the instruments would set at risk the current positive 
developments. 

We have seen in  the analysis of relevance that the market  gap on  access  to  finance for 
SMEs  is  still  an  issue  despite  the  progress  made.  According  to  LGF  intermediaries,  if 
LGF  support  were  no  longer  available,  SMEs  would  face  more  difficulties  in  getting 
access to finance. It could be expected that fewer SMEs would get a loan or that interest 
rates and collateral requirements would rise. The effect of a hypothetical termination of 
the  LGF  support  on  financial  intermediaries  would  be  a  reduction  of  activities,  both  in 
terms  of  a  lower  volume  of  (counter)  guarantees  provided  and  in  terms  of  a  reduced 
number of products offered to higher risk SMEs. 

The  EFG  intermediaries  considered  action  at  the  European  level  to  be  crucial  for  the 
European  equity  market  to  gain  maturity.  Equity  markets  in  Europe  are  still  relatively 
weak and only a few larger Member States can sustain their own equity markets.  

The EEN member organisations stated in the survey that the quality, the coverage and the 
variety  of  services  in  their  own  country  or  region  would  decrease  or  stop  if  the 
Commission would lower funding. Full regional coverage and the principle of reciprocity 
are essential for the effectiveness of the Network. Therefore, if even only a small number 
of regions would not able to continue their support offer, it would have negative impacts 
on the service delivery in other parts of the Network as well. 

On  Entrepreneurship,  there  is  still  substantial  room  for  further  improvement,  with  data 
persistently showing relatively low levels of entrepreneurial interest and activity, as well 
as high rates of failure and slow rates of growth amongst newly established firms. 

Also in the case of the other COSME action lines, the ‘benefit of their implementation at 
the European rather than national or regional levels, is very high. This is especially the 
case  for  those  actions  where  the  European  scale  is  of  critical  importance  for  the 
attainment of positive effects also at the national and regional levels. Examples of such 
actions  are  those  related  to  eSkills,  the  Tourism  actions  bringing  together  actors  in 
European  value  chains,  the  Clusters  Go  International  programme  fostering  European 
implement 
partnerships  among  clusters 
internationalisation  strategies.  Multiple  barriers  still  exist  preventing  SMEs  from 
undergoing  digital  transformation.  Considerable  differences  between  the  MS  are  still 
present  resulting  regions  lagging  behind  in  terms  of  digital  transformation.  The 
interviewed  stakeholders  agreed  that  a  campaign  such  as  WATIFY  is  needed  and  met 
market needs. Last, but not least, the SME policy actions providing strategic information 
as  well  as  network  opportunities  for  national  policy-makers,  fostering  improved  SME 
policy-making  throughout  Europe  as  well  as  an  enhanced  convergence.  The  scale  and 

to  develop  and 

in  order 

jointly 

57 REGULATION (EU) No 1287/2013 

38 

 
                                                           
depth of support that COSME offers goes well beyond the support that can be offered to 
European SMEs through national or regional initiatives. 

6.  CONCLUSIONS AND LESSONS LEARNED 

  What is working/what is not working, how it links to the intervention? 

COSME  actions  are  highly  relevant  in  addressing  the  challenges  related  to  fostering 
economic  growth  and  creating  employment  opportunities  and  is  strongly  aligned  to  the 
evolving  needs  of  SMEs.  COSME  supports  competitiveness  and  growth  of  all  SMEs. 
The  strength  of  the  programme  lies  in  its  focus on  enhancing  the  SMEs’  capacities  for 
growth and in its attention to the SMEs’ needs for concrete information and knowledge 
that are of immediate and practical use in their business practices. The programme also 
addresses in a direct and concrete manner the need of young entrepreneurs to strengthen 
their entrepreneurship skills.  

However, this  approach limits the possibility of specifically addressing other issues.  Its 
two  major  actions,  LGF  and  EEN,  accounting  for  about  80%  of  the  total  budget 
committed,  target  all  SMEs  without  making  distinctions  to  reflect  the  programme’s 
‘inclusive  and  sustainable  growth’  and  ‘global  competitiveness’  policy  objectives.  The 
two  major  actions  can  therefore  create  only  ‘unintended’  effects  in  those  areas.   As  a 
result, the relevance of  COSME for policy objectives such as  gender mainstreaming or 
climate change is limited. 

The strength of the programme lies in particular in the use of intermediaries who have a 
direct  and  longstanding  contact  with  SMEs  for  the  implementation  of  the  programme. 
COSME  exploits  the  proximity  of  these  intermediaries  to  SMEs  and  facilitates  the 
integration  of  services  provided  by  COSME  with  services  provided  by  these 
intermediaries  in  their  national  and  regional  contexts.  This  allows  for  the  provision  of 
customised SME support tailored to the needs and challenges of specific sectors, such as 
tourism,  textiles,  creative  industries  etc.  and  to  reach  a  high  multiplier  effect  of 
actions.   COSME  has  thus  a  good  level  of  European  added-value.  The  European 
dimension  constitutes  the  very  essence  of  the  design  of  COSME  actions.  Moreover, 
COSME actions have helped to enhance national,  regional and local level measures for 
SME  support  and  the  teaming  up  of  actors  across  EU  value  chains  to  boost 
internationalisation  and  industrial  modernisation.  The  feedback  from  beneficiaries  is 
generally positive with a substantial majority reporting a good cost-benefit ratio for their 
participation. 

COSME is  complementary to other EU policy instruments and programmes supporting 
business, without significant overlaps.  It is the only EU programme designed to support 
SMEs' competitiveness and growth. Its ‘non-targeted’ offering addresses the basic needs 
of enterprises, facilitating access  to  finance, fostering  cooperation and providing access 
to  advice.   It  responds  to  existing needs,  because a  financing  gap continues to  exist in 
Europe,  depending  on  the  national  context,  and  information  asymmetries,  specifically 
those  related  to  market  opportunities  in  the  European  Single  Market  and  the  global 
market,  continue  to  require  policy  intervention  for  facilitating  knowledge  transfer  and 
exchange as well as cross-border co-operation; 

The supporting study highlights several aspects to be improved: 

First,  as  stipulated  by  the  COSME  regulation,  some  80%  of  the  budget  is  used  for 
promoting  SMEs'  access  to  finance  and  access  to  markets.  The  remaining  20%  is 

39 

 
allocated between a number of smaller actions with a limited budget and with a low cost-
benefit  ratio  regarding  the  implementation.  EASME  (Executive  Agency  for  SMEs) 
implements  35%  of  the  overall  programme  budget,  with  two  flagship  COSME  actions 
(Enterprise Europe Network and Erasmus for Young Entrepreneurs) as well as a host of 
smaller  projects.  The  fragmentation  of  actions  and  limitations  in  terms  of  budget  (as 
flagged out for the EYE and cluster measures) hampers the effective implementation58.  

Second,  while  COSME  is  successful  in  fostering  economic  growth  and  the  creation  of 
employment  opportunities,  there  is  scope  for  strengthening  its  responsiveness  to  EU 
objectives  related  to  EU  global  competitiveness  and  sustainable  and  inclusive  growth. 
COSME's  main  instruments  are  designed  to  address  all  SMEs,  independently  of  their 
sector  and  industrial  specialisations.  They  only  indirectly  address  challenges  such  as 
climate change, gender mainstreaming or youth unemployment. 

Third, while there are no major overlaps with other policy initiatives, there is scope for 
creating further synergies with other EU, national and regional actions.  

Fourth,  in  relation  to  monitoring  the  evaluation  points  out  to  a  lack  of  centralised  data 
about  implementation  and  indicators  are  mostly  based  on  outputs  rather  than  on  long-
term  effects.  Some  efforts  are  still  needed  to  centralise  the  data  that  at  present  is 
dispersed  between  the  coordinating  of  the  programme,  the  units  managing  individual 
actions and the delegated entities. 

  Lessons learned: 

While  acknowledging  –  based  on  the  available  evidence  –  the  strength  of  the  COSME 
financial  instruments  to  deliver  a  financial  support  to  the  SMEs  that  most  needed  it, 
several areas for improvement are identified: 

  Better  links  between  financial  instruments  and  other  parts  of  the  future  SME 

programme; 

  Better co-ordination between financial instruments and national EU schemes; 
  Discontinuation  of  the  EUR  150 000  threshold  which  has  a  negative  impact  on 
the  guarantee  facility  and  created  significant 

the  efficient  delivery  of 
administrative burden. 

A future EU loan guarantee facility should help to ensure a more level playing field for 
SMEs in those countries where, according to current studies, the needs among SMEs are 
highest.  The  existing  Equity  Facility  for  Growth  is  considered  to  be  effective. 
Nevertheless, it appears necessary to reduce the number of financial instruments and to 
align the Equity Facility for Growth with the equity facilities established under EFSI. 

The  limitation  of  budget  beyond  the  financial  instruments  and  the  EEN  creates  a  high 
competition  between  the  remaining  actions.  It  represents  20%  of  the  budget  for  a  high 
number of actions. A fragmented budget reduces the potential impact of these actions and 
it creates a high competition in the published calls.   

58  For  example,  16  FTE  in  EASME  implemented  91  contracts  with  a  budget  of  58  million  for  the  EEN  in  2017 
compared to 36 FTEs who implement 47 actions divided into 80 contracts for a budget of 45 million. 

40 

 
 
 
 
                                                           
There is still scope for simplification.  Stakeholders report a good cost-benefit ratio for 
their  participation  in  the  programme  but  report  that  IT  systems  could  be  more  user-
friendly and there is administrative burden for participating in the programme actions. 

At the level of coherence, while actions are rather synergetic with other EU programmes 
or  complement  well  national  and  regional  instruments  for  SME  support,  there  are  still 
some overlaps. 

Finally,  the  lack  of  a  centralised  database  for  the  implementation  of  the  programme 
reduces  the  effectivity  of  monitoring  and  evaluation  as  well  as  the  visibility  of  the 
programme.  

  Does actual performance meet expectations? 

in 

COSME 
the  period  2014-2016  produced  positive  results.  The  programme 
implementation is on track to reach the objectives set out in the legal basis by the end of 
2020.  

The COSME LGF has been successfully set-up and is fully on track to delivering on the 
targets  set  in  terms  of  SMEs  to  be  financed  and  volume  of  financing  to  be  made 
available. It confirms that the "COSME actions fully respond to the SMEs’ current needs 
to access finance". 

The EYE programme has been established as pilot project in 2008 under CIP and since 
2014  incorporated  under  COSME.  It  has  been  particularly  effective  in  reaching  its 
knowledge- and skills-related objectives and has laid the ground for important outcomes 
in  terms  of  business  relationships  and  internationalization  for  the  participating 
entrepreneurs.  However,  the  scale  of  the  activities  (in  total  about  2,000  new 
entrepreneurs)  poses  limits  to  the  importance  of  this  action  line  within  the  overall 
programme portfolio. 

The limited evidence available suggests that COSME will have a positive impact on its 
beneficiaries’  growth  and  employment.  The  EIF  has  an  essential  role  in  the 
implementation  of  the  financial  instruments  and  EASME  plays  an  important  and  much 
appreciated role in creating learning opportunities for the EEN members to enhance their 
capacities and better serve the SMEs in their local environments. 

The EEN is very relevant to the overall need of SMEs in building trust in finding suitable 
partners. However, it does not address the barrier of high costs that come with building 
trust  in  finding  suitable  partners.  As  a  result,  a  potential  upgrade  of  the  programme 
could  therefore  include  other  types  of  support,  such  as  demonstration  projects, 
feasibility studies and knowledge-acquisition activities.  

COSME  meets  the  expectations  in  terms  of  relevance,  efficiency,  effectiveness  and 
coherence. Its actions show a high level of EU added-value. However, some efforts are 
still  needed  to  improve  efficiency,  effectiveness  and  coherence.  Some  issues  already 
pointed out by the CIP and EIP evaluations have not been solved during the 2014-2016 
period:  the  fragmentation  of  the  programme  into  smaller  actions  and  the  data 
management.   

  What issues need to be addressed or will it resolve over time? 

41 

 
 
COSME  covers  a  broad  range  of  actions  to  implement  the  objectives  set  out  in  the 
COSME  regulation  with  a  limited  budget.  A  clearer  definition  of  the  COSME  priority 
areas should enable the  programme to  reach more efficiency and  effectiveness.  Several 
possibilities could be envisaged:  

1) Deploying its existing resources in a more focused / strategic manner;  

2) Entering new strategic partnerships with other EU programmes by strategically 
sharing the responsibility for interventions in specific areas; and  

3)  Increasing its overall budget  in  order to  allow it to  run both  the non-targeted 
approach and strategic programmes in parallel. 

Before 2020, an increase in the overall budget is not a valid option.  The MFF review did 
not retain this possibility, except for the financial instruments. 

A more strategic prioritisation of objectives in the design of the work-programmes could 
be  implemented  for  the  remaining  years  until  the  end  of  the  programme  in  2020.  This 
could help reduce the fragmentation of the budget. A more centralised approach to data 
management  would  ensure  higher  efficiency  in  the  programme  management  and  a 
strengthened  capacity  for  a  quality  implementation  of  the  monitoring  and  evaluation 
function.  It  would  also  pave  the  way  for  the  creation  of  the  monitoring  system  for  the 
post-2020 period. 

For the design of the post-2020 period successor to COSME, an ambitious scenario could 
be  proposed  in  the  impact  assessment  of  the  successor  programme  to  strengthen  the 
overall effectiveness of the programme. However, considering the potential reduction of 
the overall EU budget due to the BREXIT and the new challenges the EU has to tackle, 
(such as Defence and Security), the most realistic option seems to be to better exploit the 
available budget, and focus on the most successful actions and reserve a budget for new 
ideas  and  pilots.  Synergies  with  other  programmes  could  be  also  exploited  in  order  to 
really  focus  on  the  actions  where  COSME  is  the  most  effective,  and  to  support 
enterprises in all stages of their life-cycle, from their creation to their growth.  

At  this  stage,  substantial  changes  in  the  financial  instruments  or  the  Enterprise  Europe 
network  will  be  counter-productive  to  implement.  Considering  the  sophisticated 
implementation chain and the respective concomitant amendments procedures needed, it 
is  reasonable  to  expect  that  any  tangible  impact  would  only  produce  effect  after  2020.  
The potential burden imposed on intermediaries and SMEs following any changes should 
not  be  underestimated  either.  It  is  more  realistic  to  accept  that  the  conclusions  of  the 
evaluation will rather feed the design of the post-2020 instruments. 

However,  in  the  impact  assessment  for  the  next  multi-annual  financial  framework,  the 
Commission proposes: 

  To  improve  the  internal  coherence  of  different  parts  of  the  programme,  better 
linkages will be established by further strengthening the existing cooperation with 
the  Enterprise  Europe  Network59  (use  the  Enterprise  Europe  Network  as  a 
stronger signposting element); 

59 https://een.ec.europa.eu/ 

42 

 
 
                                                           
  A  better  upstream  co-ordination  between  financial  instruments  for  SMEs 

 

established by Member 
 States and those established under the SME window of the InvestEU Fund will 
be  sought  by  using  the  existing  SME  Envoys  Network60  as  an  information 
exchange forum;  

  To  reduce  administrative  burden  for  SMEs  and  financial  intermediaries  and  to 
improve  the  impact  of  the  SME  guarantee  facility,  the  EUR  150  000  threshold 
will be eliminated. 

This  interim  evaluation  will  pave  the  way  for  the  final  evaluation  of  COSME  that  will 
also feed the detailed design of the first work programmes of the period post 2020.  The 
final  evaluation  should  analyse  more  in  detail  the  wide  variety  of  COSME  actions  to 
provide the Commission with data that will support the strategy to select the actions that 
will  compose  the  future  COSME  in  the  next  multi-annual  financial  framework. 
Preliminary findings of the interim evaluation have informed the impact assessments on 
Single  Market  and  Invest  EU  programmes  for  the  next  Multi-annual  Financial 
Framework 2021-2027. 

60 http://ec.europa.eu/growth/smes/business-friendly-environment/small-business-act/sme-envoys_en 

43 

 
 
 
 
 
                                                           
Annex 1: Procedural information 

1.  LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

LEAD DG: GROW 

DECIDE PLANNING: PLAN_2016_223 

CWP: 

2.  ORGANISATION AND TIMING 

An inter-service steering group was created in Q4 2016.  The steering group was 
constituted of the following DGs: 

  BUDG 
  SG (A.2 (ex C.1)) 
  ENV 
  EMPL 
  REGIO 
  RTD 
  TRADE 
  ECFIN 
  ENER 
  EAC 

In  total  7  meetings  of  the  Interservice  steering  group  (ISSG)  have  been  organised 
between Q4 2016 and December 2017. 

After having been consulted with  the  ISSG and approved by the secretariat general  the 
roadmap was published the 16/01/2017.  No feedback was received from stakeholders. 

The  consultation  strategy  and  the  Terms  of  reference  for  the  supporting  study  were 
submitted to the ISSG during Q4 2016. The supporting study was concluded by using a 
framework  contract  with  Technopolis  Group.  The  Specific  contract  was  signed  on 
19/12/2016 and the contract started in January 2017. 

The inception meeting took place on 27/02/17 and subsequently the inception report was 
submitted to the ISSG. The inception report was approved on 01/06/2017.   

The  interim  report  was  submitted  on  24/05/2017  and  served  to  prepare  the  draft  final 
report  that  was  received  on  04/10/2017.  It  was  presented  to  the  ISSG  (including 
presentation by the contractor) at the 7th meeting of 13/10/17. The comments from DG 
GROW  and  other  members  of  the  group  have  been  transmitted  to  the  contractor  to 
prepare  the  final  report.  From  the  ISSG,  GROW,  ECFIN,  REGIO  and  SG  requested 
changes. 

The  Final  invoice  was  received  the  23/10/17.  Contract  report  approval  was  suspended 
20/10/17  in  order  to  consider  ISSG  comments.  The  Final  Report  was  received 

44 

 
 
20/11/2017.    It  was  sent  to  the  members  of  the  ISSG  with  deadline  of  27/11/17  for 
comments or approval. 

Although comments and corrections had been taken into account by the contractor, there 
were  still  corrections  and  modifications  made  and  requested  by  the  ISSG  prior  to 
finalising  the  document.  The  revised  final  report  was  received  the  08/12/17.  On  12-
13/12/17  the  approval  of  SG,  ECFIN,  TRADE  and  REGIO  was  received,  no  other 
member  of  the  ISSG  has  made  any  objection  so  the  final  report  was  considered  as 
approved by the ISSG and the final report was sent to all members of the group. 

The final versions of all deliverables received from the contractor including IPR annexes 
from FWC are registered in ARES (ARES(2017)6154624 of 14/12/17).   

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

none 

4.  CONSULTATION OF THE RSB (IF APPLICABLE) 

not applicable 

5.  EVIDENCE, SOURCES AND QUALITY 

The major difficulty for this evaluation has been the lack in data to measure results and 
impact.  By  the  time  of  evaluation,  only  major  and  long-standing  actions  produced 
sufficient  data  to  allow  for  measuring  results.  The  evaluation  measured  on  basis  of 
outcomes and outputs produced the probability for the programme's actions to reach the 
desired effects. The main evidence for this evaluation is the supporting study performed 
by Technopolis Group. The quality of the supporting study has been positively assessed 
by  the  ISSG  and  other  limitations  are  indicated  in  the  synopsis  and  methodological 
report.  

The supporting study took into account the below evidence, together with evidence from 
the evaluation of the IPR SME helpdesks. The complete list of sources is indicated in the 
methodological report.  

  Regulation  (EU)  1287/2013  of  the  European  Parliament  and  of  the 
Council  of  11  December  2013  establishing  a  Programme  for  the 
Competitiveness  of  Enterprises  and  small  and  medium-sized 
enterprises  (COSME)  (2014  -  2020)  and  repealing  Decision  No 
1639/2006/EC.61 

  Decision 1639/2006/EC of the European Parliament and of the Council of 
Innovation 

24 October 2006  establishing  a  Competitiveness  and 
Framework Programme (2007-2013)62 
Impact  Assessment  for  the  proposal  for  a  COSME  regulation  COM 
(2011)83463 

 

61 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1287 

62 http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=URISERV%3An26104 

45 

 
 
 
 
 
 
                                                           
  MFF review64 
  Ex-ante  Preparatory  documents 

consultation65 

for  COSME, 

including  public 

  CIP Impacts and Results, including evaluations66 
  EACI evaluations67 
  Final evaluation of the Enterprise Europe Network 2008 – 201468  
  SME Performance Review and Small Business Act Factsheets69  
  The  Small  Business  Act  for  Europe,  including  the  Report  on  the  2014 

public consultation70  

  Study commissioned by the Committee of the Regions: Implementation of 
the Small Business Act for Europe (SBA) and Entrepreneurship Policies at 
Local and Regional Level71 

  The Single Market Strategy72 
  Public consultation under the Start-up Initiative73 

  The Single Market Forum74 
  SAFE (Survey on Access to Finance)75  
  Flash Eurobarometer 354, ENTREPRENEURSHIP IN THE EU AND 

BEYOND, 08/201276  

  Flash Eurobarometer on the internationalization of SMEs77:  

63http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52011SC1452 

  and  http://eur-lex.europa.eu/legal-

content/EN/TXT/?uri=CELEX:52011SC1453 

64 http://ec.europa.eu/budget/mff/figures/index_en.cfm#com_2016_603 

65 http://ec.europa.eu/cip/public_consultation/index_en.htm 

66 http://ec.europa.eu/cip/documents/index_en.htm 

67http://bookshop.europa.eu/en/search/?webform-

id=WFSimpleSearch&DefaultButton=findSimple&WFSimpleSearch_NameOrID=EACI&SearchConditions=&Sear
chType=1&SortingAttribute=LatestYear-desc&findSimple.x=0&findSimple.y=0 

68http://bookshop.europa.eu/en/final-evaluation-of-the-impact-of-the-enterprise-europe-network-2008-2014-

pbET0415831/  Full  Report  http://bookshop.europa.eu/en/final-evaluation-of-the-impact-of-the-enterprise-europe-
network-2008-2014-pbET0415830/ 

69 http://ec.europa.eu/growth/smes/business-friendly-environment/performance-review_en 

70 https://ec.europa.eu/growth/smes/business-friendly-environment/small-business-act_en 

71http://cor.europa.eu/en/takepart/eer/Documents/SBA%20implementation%20at%20regional%20and%20local%20le

vel/EN.pdf 

72 https://ec.europa.eu/growth/single-market/strategy_en 

73 http://ec.europa.eu/growth/tools-databases/newsroom/cf/itemdetail.cfm?item_id=8723 

74 https://ec.europa.eu/growth/single-market/forum_en 

75http://ec.europa.eu/growth/access-to-finance/data-surveys_en 

http://www.ecb.europa.eu/stats/money/surveys/sme/html/index.en.html 

76 http://ec.europa.eu/public_opinion/flash/fl_354_en.pdf 

and 

77http://ec.europa.eu/COMMFrontOffice/PublicOpinion/index.cfm/Survey/getSurveyDetail/instruments/FLASH/surve

yKy/2090 

46 

 
                                                                                                                                                                            
COSME web page78  

  EASME web page79  
  EEN web page80 
  EYE (Erasmus for Young Entrepreneurs) web page81  
  EIF, Loan Guarantee Facility (including implementation status)82  
  EIF, Equity Facility for Growth83:  
  Statistics, data and studies on Tourism84  
  Data on regional funds: 

-  The ex post evaluations of the 2007-2013 ERDF and Cohesion Fund 

programmes: http://ec.europa.eu/regional_policy/en/policy/evaluations/ec/2007-
2013/#1  

-  ESF and its support to entrepreneurship and micro-finance 

(http://ec.europa.eu/esf/main.jsp?catId=3 )  

-  For the EAFRD: http://ec.europa.eu/agriculture/evaluation/rural-development-

reports/index_en.htm  

-  For the EMFF: http://ec.europa.eu/fisheries/cfp/eff/index_en.htm and 

http://ec.europa.eu/fisheries/cfp/emff/index_en.htm   

- 

as regards the 20014-2020 programmes:  

o  The expected outcomes, planned funding, etc. for all 5 ESI Funds: 

https://cohesiondata.ec.europa.eu/ (see for instance the search by SME 
competitiveness: https://cohesiondata.ec.europa.eu/themes/3 ).  
http://ec.europa.eu/regional_policy/en/policy/evaluations/data-for-research/  

o  ERDF financial instruments: 

http://ec.europa.eu/regional_policy/en/funding/financial-instruments/  

78 https://ec.europa.eu/growth/smes/cosme_en 

79 https://ec.europa.eu/easme/en 

80 http://een.ec.europa.eu/ 

81 http://www.erasmus-entrepreneurs.eu/ 

82 http://www.eif.org/what_we_do/guarantees/single_eu_debt_instrument/cosme-loan-facility-growth/ 

83 http://www.eif.org/what_we_do/equity/single_eu_equity_instrument/cosme_efg/index.htm 

84 https://ec.europa.eu/growth/tools-databases/vto/ 

47 

 
 
                                                           
Annex 2: Stakeholder consultation 

Three types of consultations were conducted: the open public consultation, the ‘quantitative’ 
targeted consultations, and the ‘qualitative’ targeted consultations. In addition, 121 interviews 
were  conducted  and  carried  out  seven  quantitative  consultations,  seven  qualitative 
consultations and one public consultation.  

The European Commission implemented the questionnaire for the public consultation on the 
“Your  Voice  in  Europe”  website  (https://ec.europa.eu/info/consultations/interim-evaluation-
programme-competitiveness-enterprises-and-small-and-medium-sized-enterprises-cosme-2014-
2020_en) on 10 May 2017 until the 31 of August 2017. 

Both  intermediaries  and  beneficiary  SMEs  were  approached  by  targeted  consultations.  In 
total,  4,100  responses  were  received.  The  response  rates  among  intermediaries  was  about 
45%;  it  was  between  10%  and  20%  among  SMEs.  The  public  consultation  received  195 
responses and 14 position papers. 

For WATIFY, The stakeholder consultation strategy included three main activities:  

1.  An on-line target survey to the main beneficiaries of the awareness raising campaign 

2.  A  general  on-line  survey  targeting  SMEs  covering  16  Member  States  conducted 

through online panels 

3.  45 interviews covering all types of direct stakeholders  

48 

 
 
 
 
 
Synopsis Report 

1 

2 

OVERVIEW OF THE CONSULTATION STRATEGY ........................................................... 50 

STAKEHOLDER GROUPS INVOLVED IN THE CONSULTATIONS ......................................... 51 

2.1 
2.2 

2.3 
2.4 

Stakeholders interviewed .................................................................................... 51 
Stakeholders involved in the qualitative targeted consultations ........................................... 52 
Results for SO1 – Access to finance ........................................................... 52 
2.2.1 
2.2.2 
Results for SO3 – Favourable environment ................................................... 53 
Stakeholders involved in the quantitative targeted consultations ......................................... 53 
The public consultation ...................................................................................... 54 
Overview ........................................................................................ 54 
2.4.1 
Number and profile of the participants ........................................................ 54 
2.4.2 

3 

ROBUSTNESS OF THE CONSULTATION RESULTS .......................................................... 55 

3.1 
3.2 
3.3 

The quantitative targeted consultations ..................................................................... 55 
The qualitative targeted consultations ...................................................................... 55 
The public consultation ...................................................................................... 56 

4 

SUMMARY OF THE RESULTS .................................................................................... 56 

4.1 
4.2 
4.3 
4.4 
4.5 

Relevance .................................................................................................... 56 
Effectiveness ................................................................................................. 56 
Efficiency .................................................................................................... 57 
Coherence .................................................................................................... 58 
European added value ....................................................................................... 58 

49 

 
 
 
 
OVERVIEW OF THE CONSULTATION STRATEGY 

Three  types  of  consultations  were  conducted:  the  open  public  consultation,  the  ‘quantitative’  targeted 
consultations, and the ‘qualitative’ targeted consultations. In addition, 121 interviews were conducted and 
carried  out  seven  quantitative  consultations,  seven  qualitative  consultations  and  one  public  consultation 
(Table 2). 

Both  intermediaries  and  beneficiary  SMEs  were  approached  by  targeted  consultations.  In  total,  4,100 
responses  were  received.  The  response  rates  among  intermediaries  was  about  45%;  it  was  between  10% 
and 20% among SMEs. The public consultation received 195 responses and 14 position papers. 

Table 2: Tools and methods used in the evaluation 

1. 

4. 

9. 

1. 

Consultation approach 

Qualitative consultations 

Quantitative consultations 

Source: Technopolis 

2. 

5. 

7. 

10. 

12. 

Tools  

Interviews 

Qualitative targeted consultation 

Targeted consultation  

Public consultation 

3. 

6. 

8. 

11. 

13. 

Number 

121 

8 

7 

1 

A  major  purpose  of  the  high-level  interviews  was  to  collect  background  and  context  information  on 
facilitators and barriers for the attainment of the expected effects as well as the eventual unintended effects. 
Wherever possible, the interviews were conducted face-to-face. Interviews with EU officials took place in 
the beginning of the study to reach an improved understanding of the overall context to the programme and 
the processes for the programme design. 

The ‘classical’ targeted consultations  were implemented in cases  where  the population was  higher than 
100  and  would  therefore  allow  for  meaningful  and  robust  statistical  data  analyses.  The  questionnaires 
mostly included close-ended questions and only a few open-ended ones. The responses were analysed by 
means  of  descriptive  statistics,  in  several  cases  also  including  cross-analyses.  We  foresaw  seven 
quantitative surveys in the Inception Report; they were all implemented. 

The  public  consultation  was  a  12-week  internet-based  questionnaire.  The  objective  was  to  cover  all 
potential relevant stakeholders, in particular those that were not targeted through the targeted consultations 
and interviews, and to collect feedback on the COSME programme and its objectives. The questionnaire 
for  the  public  consultation  aimed  at  reaching  all  types  of  stakeholders.  Given  that  the  targeted 
consultations focused exclusively on the specific objectives, it was agreed to use the public consultation to 
collect data that would provide information on the programme as a whole. 

Table  3  shows  the  evaluation  questions  covered  by  the  different  consultation  methods.  The  consultation 
questionnaires allowed for a proper triangulation of the evidence collected and helped mitigating the risk of 
lack  in  data  availability.  For  each  evaluation  question,  the  analysis  was  based  on  multiple  sources  and 
methods 

Table 3: Use of the consultations per evaluation question 

14. 

Evaluation question 

18. 
20. 

24. 
28. 

30. 
33. 

Q1.1) Programme implementation and evolution since 2014  
Q2.1) Ongoing relevance of the objectives 

Q2.2) Relevance to the stakeholders and the EU citizen 
Q2.3) Adaptation of the objectives to the political priorities  

Q3.1) Progress towards achieving the specific objectives 
Q3.2) Factors driving or hindering progress  

50 

e
i
v
r
e
t
n
I

.
5
1

e
v
i
t
a
t
i
l
a
u
q
&

s
w

s
n
o
i
t
a
t
l
u
s
n
o
c

d
e
t
e
g
r
a
t

a
t
i
t
n
a
u
Q

.
6
1

d
e
t
e
g
r
a
t

e
v
i
t

n
o
i
t
a
t
l
u
s
n
o
c

19. 
21. 

25. 
29. 

31. 
34. 

X  
X 22. 

X 26. 
X  

X 32. 
X 35. 

c
i
l
b
u
P

.
7
1

n
o
i
t
a
t
l
u
s
n
o
c

X 23. 

X 27. 

X  
X  

X 

X 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36. 

39. 

42. 
45. 
48. 
52. 
56. 
60. 
64. 
2. 

Q3.3) Progress towards contributing to the 5 Union priorities  

Q4.1) Regulatory/administrative costs and benefits for the different stakeholders 

Q4.2) Costs (direct and indirect) proportionate to the benefits  
Q4.3) Efficiency of the programme implementation structure and governance  
Q5.1) Internal coherence 
Q5.2) Coherence with ESI funds, H2020, EFSI, EASI and other EU programmes 
Q5.3) Coherence with national programmes 
Q6.1) Added value of implementation at the EU level 
Q6.2) Ongoing need for action at EU level and why 
Source: Technopolis 

37. 

40. 

43. 
46. 
49. 
53. 
57. 
61. 
65. 

X  

X 41. 

X 44. 
X 47. 
X 50. 
X 54. 
X 58. 
X 62. 
X 66. 

38. 

X  

X  
X  
X 51. 
X 55. 
X 59. 
X 63. 
X 67. 

X 

X 
X 
X 
X 

The  time  plan  of  the  consultation  strategy  was  that  the  targeted  consultations  would  take  place  in  the 
month of April, ending in the beginning of May. The statistical analyses of the survey data were scheduled 
for the month of May, in time for the evaluation team to start drafting the internal reports at the thematic 
area level in the month of June. This strict time schedule was dictated by the need of the evaluation team to 
run the triangulation process, eventually conduct additional interviews, and subsequently, various runs of 
the aggregation of the evaluation findings (at thematic area and programme level), in time for the delivery 
of the draft final report in the beginning of September. 

Table 4 presents an overview of the public consultation process, from start to finish.  

Table 4 Overview the open public consultation process 

68. 
70. 
72. 
74. 
76. 
78. 
80. 
82. 

85. 
3. 

Activity 
Phone call with the EC  
Delivery of first draft questionnaire 
Review from the EC 
Validation of final questionnaire 
Editing in EN by EC 
Translation in other EU languages 
Deadline of the consultation 

EC to send inputs to Technopolis Group 
Source: Technopolis 

69. 
71. 
73. 
75. 
77. 
79. 
81. 
83. 
84. 
86. 

Timing 
10 February 2017 
24 March 2017 
29 March 2017 
26 April 2017 
26 April – 5 May 2017 
6 May – 12 June 
Launch date: 10 May 
Closing date: 31 August  
Latest results received mid-September (13/09/2017) 

STAKEHOLDER GROUPS INVOLVED IN THE CONSULTATIONS 

Stakeholders interviewed 

Overall, the contractors' team conducted 121 interviews. These were distributed over the SOs as shown in 
Table 5, below.  

Table 5: Stakeholders interviewed 

87. 

 SO 

88. 

 Interviewed stakeholders 

90. 
SO1 – 
Access to finance 

91. 
and EIB) 

Representatives from EU institutions (DG GROW, DG REGIO, DG RTD, DG ECFIN, EIF 

93. 

intermediary and industry organisations 

96. 
fostering coherence within the Network. 

EU officials who were responsible for the Network and for other programmes aimed at 

95. 
SO2 – 
Access to market 

98. 

Network members 

100. 
intermediaries at a higher aggregated level. 

Umbrella organisations were also interviewed so as gain the views of SMEs and 

SO3 – 

102. 
Favourable 
environment 

111. 
SO4 – 
Entrepreneurship 

103. 
105. 

107. 

EU officials in DG GROW, EASME and other DGs 
Beneficiaries (cluster managers) 

National government representatives 

109. 

Umbrella organisations and national experts 

112. 
Individuals responsible for the design, management and oversight of COSME 
entrepreneurship actions – including European Commission officials, EASME staff and individuals 
within the EYE support office 

114. 

116. 

Host Entrepreneurs – participants in and beneficiaries of the EYE programme 

New Entrepreneurs - participants in and beneficiaries of the EYE programme 

51 

89. 

92. 

94. 

97. 

99. 

101. 

104. 
106. 

108. 

110. 

113. 

115. 

117. 

 Nr 

15 

6 

9 

19 

5 

16 
6 

4 

10 

20 

5 

6 

 
 
87. 

 SO 

88. 

 Interviewed stakeholders 

89. 

 Nr 

4. 

Source: Technopolis 

Stakeholders involved in the qualitative targeted consultations  

Table 6 Overview of the participation in the qualitative surveys 

118. 

SO 

119. 

Actors & stakeholders targeted 

SO1 

123. 
– Access to 
finance 

SO3 

140. 
– Favourable 
environment 

LGF intermediaries 
EFG intermediaries 
EFG SMEs 
Members of European Intermediary 

124. 
128. 
132. 
136. 
Organisations & LGF sub-intermediaries 
141. 

SME Policy: SME Envoys 

145. 
members 

SME Policy: SPR Working Group 

149. 
Sectoral competitiveness: European 
stakeholder organisations, national stakeholder 
organisations (associations), national and regional 
public authorities   
153. 
organisations, national stakeholder organisations 
dealing with e-skills 

E-skills: European stakeholder 

120. 
sample 
125. 
129. 
133. 

137. 

Target 

21 
5 
12 

6 

28 
127 

142. 
146. 
(according to the email 
list received from EC)  

121. 
responses 
126. 
130. 
134. 

138. 

143. 

147. 

Nr of 

16 
3 
4* 

5 

15 

23 

122. 
rate 
127. 
131. 
135. 

139. 

144. 

148. 

Response 

76% 
60% 
33% 

83% 

54% 

18% 

150. 

72 

151. 

7 

152. 

10% 

154. 

31** 

155. 

8     

156. 

26% 

5. 
valid. Out of them 6 contacts were contacted for interviews and 31 for the qualitative survey.  

Source: Technopolis - * conducted through a cascade approach **out of the 53 e-mail addresses received, only 37 were 

Results for SO1 – Access to finance 

Among the 61 financial intermediaries receiving LGF funds, 20 were approached. Four qualitative surveys 
were conducted: 

1  A  written  questionnaire  was  sent  via  e-mail  to  21  LGF  intermediaries,  asking  the  recipients  to 
complete  the  questionnaire  within  two  weeks.  Five  of  them  were  the  financial  intermediaries  (cat  1 
promotional  banks  and  cat  2  guarantee  societies)  and  others  were  selected  according  to  their  type 
(Promotional  Institution,  Guarantee  Institution,  Commercial  Bank,  Leasing  Company),  country  of 
establishment, direct/counter Guarantees and risk category in order to have a wider and representative 
possible  coverage.  Some  of  them  where  involved  also  in  the  CIP  SMEG  and  overall  50%  were 
currently using also InnoFin SMEG; they gave us more insights on the process and leverage effect.  

Two reminders were sent to LGF intermediaries that did not respond by the fixed deadline. Finally, the 
team  received  16  completed  questionnaires,  corresponding  to  a  response  rate  of  76%.  One  LGF 
intermediary  explicitly  refused  to  participate,  while  four  intermediaries  did  not  answer.  Eight 
responding  intermediaries  provided  direct  guarantees  for  loans  and  another  eight  intermediaries 
provided counter guarantees. 

2  A written survey of EFG intermediaries was conducted. The team originally planned to collect data 
from  five  EFG  intermediaries.  The  responding  EFG  intermediaries  make  equity  and  equity-related 
investments in growth-oriented start-ups and SMEs all over Europe. Only three out of five contacted 
intermediaries completed the written questionnaire.  

3  The  EFG  intermediaries  were  asked  to  cascade  the  survey  to  selected  SMEs85  that  received  equity 
funding  from  the  EFG  instrument.  It  was  planned  that  12  SMEs  were  to  be  contacted  by  their 
respective intermediaries.  Due to the limited response from the intermediaries, only a limited number 
of  SMEs  that  were  beneficiaries  of  the  EFG  (EFG  SMEs)  could  be  reached.  As  a  result,  the  team 
received only four completed questionnaires (out of 12) from EFG SMEs. 

4  Five of the six contacted European Intermediary Organisations were asked by the team to cascade 
the  survey  to  their  members.  Except  for  one  intermediary,  the  European  intermediary  organisations 
contacted their members prior to the interviews and collected information on their members’ opinions 
and  views.  In  two  cases,  the  interviews  were  even  conducted  in  the  form  of  phone  conferences  in 
which  representatives  of  both  the  umbrella  organisation  and  individual  members  of  the  intermediary 

85 the ones which received EFG investment in the first half of 2016 

52 

 
                                                           
participated. In this way, the opinions and views expressed during the interviews directly reflect those 
of individual members. 

Results for SO3 – Favourable environment 

We  launched  a  total  of  four  ‘qualitative’  surveys  related  to  the  actions  under  SO3.  A  first  step  in  the 
process  was the drafting of the  survey questionnaires.  Table 7 below,  gives an overview of this process, 
which was rather time consuming due to the number of surveys to be covered. 

Table 7: Overview of status of qualitative surveys 

157. 

Survey  

158. 

Date of draft  

159. 

Approval date 

160. 

Launch of survey 

SME Policy – SME 

SME Policy – SPR 

161. 
Envoys 
165. 
Working Group 
169. 
173. 
competitiveness  
6. 

E-skills  
Sectoral 

Source: Technopolis 

21.03.2017  - draft 1; 

162. 
27.03.2017 – draft 2 
166. 

27.03.2017 

170. 
174. 

10.04.2017 
10.04.2017 

163. 

167. 

171. 
175. 

27.03.2017 

11.04.2017 

10.04.2017 
18.04.2017 

164. 

168. 

172. 
176. 

28.03.2017 

11.04.2017 

28.04.2017 
02.05.2017 

The results of these surveys vary substantially. There was a good level of response to the survey in the field 
of SME Policy (perhaps due to the fact that it was sent out by the European Commission), while response 
rates were low for e-skills and sectoral competitiveness. Even though the contacted actors had participated 
in  events  or  activities  of  the  COSME  actions  in  the  field  of  KETs,  e-skills,  or  sectoral  competitiveness, 
some  of  them  mentioned  that  they  were  not  aware  of  the  COSME  programme  nor  had  they  had  any 
interaction with it. In addition, despite being contacted several times, the members of the Digital Coalition 
for  Skills  and  Jobs  (to  whom  the  e-skills  survey  was  targeted)  were  not  responsive  –  only  the  Coalition 
members contacted for interviews were willing to contribute.  

Stakeholders involved in the quantitative targeted consultations 

Table  8  presents  an  overview  of  the  quantitative  surveys,  including  the  target  group,  population,  target 
sample, responses, expected response rates and response rates. Out of the seven consultations, five reached 
or exceeded the expectations in terms of response rate. One did not reach its targets: the survey to the EEN 
member organisations.  

Table 8: Overview – Quantitative surveys  

177. 

# 178. 
ction 

A

1 

2 

3 

4 

5 

Loan 
Guarantee 
Facility 
Enterprise 
Europe 
Network 
(EEN) 
Enterprise 
Europe 
Network 
(EEN) 
Clusters 
Go 
Internatio
nal  
Tourism 
(Calls) 

179. 
ype of 
stakeholde
r 
Beneficia
ry SMEs 

T

180. 
op. 

T

P

181. 
argeted 
sample 

N

182. 
umber of 
responses 

E

183. 
xpected 
response rate 

184. 
R
esponse rate 
reached 

C

185. 
losing date 
in 2017 

140,000 

3,875 

356* 

6.3%  

9% 

18 July 

523** 

530 

270 

~60% 

48% 

17 May 

100,000 

50,000 

2,334 

~5% 

5% 

17 May 

136 

136 

48 

25-30% 

35% 

5 May 

377 

389 

162 

20% 

41% 

10 May 

 EEN 
member 
organisat
ions 
Beneficia
ry SMEs  
(EEN 
clients) 
Cluster 
managers 

Beneficia
ry SMEs 
and 
Intermed
iary 
agencies 
/ 
organisat
ions  

53 

 
6 

7 

7. 

595 

25% 

2,117 

2,343 

New 
Entrepre
neurs 

Erasmus 
for Young 
Entrepren
eurs 
Erasmus 
for Young 
Entrepren
eurs 
Source: Technopolis - *Reached via 20 financial intermediaries. **Reached via 92 coordinators. 

Host 
Entrepre
neurs 

1,800 

1,620 

10% 

245 

28% 

15% 

5 May 

5 May 

The public consultation 

Overview 

The  approach  of  the  public  consultation  followed  the  consultation  process  defined  by  the  European 
Commission (Better Regulation Toolbox/Tool #50).  The questionnaire was available in all EU languages. 
The European Commission implemented the questionnaire for the public consultation on the “Your Voice 
in Europe” website and the questionnaire was available in English on 10 May 2017. It remained accessible 
for  12  weeks,  starting  from  the  date  on  which  the  last  translation  was  made  available  online.  The 
dissemination  of  the  public  consultation  fell  under  the  responsibility  of  the  European  Commission. 
EASME  and  the  EC’s  communication  units  advertised  the  public  consultation  on  COSME,  EASME  and 
EIF  websites;  through  EEN  networks  and  relevant  national  contact  points;  through  other  agencies  and 
networks under COSME; and via other EC social media. 

Number and profile of the participants 

The public consultation  was  closed on the 31  August 2017.  A  total of  195 responses  were received, as 
well as 14 position papers. The survey responses were analysed using descriptive statistics. 

The  questionnaire  also  included  questions  that  allowed  to  profile  the  respondents  to  the  public 
consultations. Half of the respondents were active in industry/business; the second largest group comprised 
of  financial  intermediaries  (financial  institutions  or  finance  providers).  The  respondents  under  ‘others’ 
included  law  firms,  research  centres,  innovation  centres,  NGOs  and  associations;  business  organisations; 
public authorities; and citizens (see Figure 4). Five countries account for two thirds of the answers (France, 
the UK, Italy, Spain and Germany).  

Figure 4 Public consultation - profile of the participants 

Source: COSME public consultation, 2017, n=195 

Two-third of the respondents, i.e. 88 respondents, indicated that their organisation had benefitted from the 
COSME  programme.  Five  respondents  were  involved  in  more  than  one  action  (e.g.  EEN  as  a  member 
organisation and beneficiary, etc.). The Loan Guarantee Facility (LGF) is, by far, the most recurrent action. 

Table 9 Number of respondents per type of financial support 

186. 

Type 

187. 

Action 

189. 
finance 

Access to 

190. 

Loan Guarantee Facility 

54 

188. 
beneficiaries 
191. 
192. 
193. 

Number of 

55 in total: 
29 companies 
16 financial institutions 

 
 
186. 

Type 

187. 

Action 

197. 
markets 

202. 

207. 
environment 

212. 

8. 

Access to 

Favourable 

Entrepreneurship 

195. 
Equity Facility for Growth 
198. 
Enterprise Europe Network (EEN) – EEN member organisations 
200. 
Enterprise Europe Network (EEN) – beneficiaries 
203. 
Erasmus for Young Entrepreneurs – New Entrepreneurs 
205. 
Erasmus for Young Entrepreneurs – Host Entrepreneurs 
208. 
Cluster internationalisation 
210. 
Tourism (calls) 
213. 
Other: 
214. 
Selected as an expert 
215. 
NCP for COSME including financial instruments and publishing 
calls for proposals in all areas / Coordinator EEN including extra support for 
EYE-participation 
216. 
COS-DESIGN 
Source: COSME public consultation (n=88) 

Number of 

188. 
beneficiaries 
194. 
196. 
199. 
201. 
204. 
206. 
209. 
211. 

10 others 
5 
15 
13 
4 
4 
2 
2 

217. 
218. 

3 

ROBUSTNESS OF THE CONSULTATION RESULTS 

A key method to ensure robustness of the analysis in an evaluation is the triangulation of the findings, i.e. 
the validation of information collected through one method by comparing it with the information collected 
through  another  method.  In  this  section,  instead,  we  expand  on  the  robustness  of  the  findings  collected 
through the single consultation methods. 

The quantitative targeted consultations 

Apart of the  more than satisfactory response rates reached for the quantitative targeted  consultations, the 
profile of the responding organisations is also in close to all surveys very similar to the target sample, thus 
guaranteeing the statistical representativeness. Exceptions to the rule are the survey of the Network client 
SMEs, managed through a cascade approach, for which less responses than expected were reached; in this 
case,  also  the  representativeness  of  the  responses  cannot  be  checked.  Another  exception  is  the  survey 
related  to  the  Clusters  Go  International  programme  where  more  funded  ESCPs  than  non-funded  ones 
responded compared to the target sample. In this case, however, this does  not imply  forcefully a lack of 
representativeness for the funded ESCPs. 

Table 10: Criteria and results regarding the statistical representativeness analysis 

219. 

Survey 

220. 
rate 

Response 

Criteria 

221. 
for 
representativeness 

Country of residence 
Sector of activity 

224. 

9% 

226. 

48% 

Country grouping  
Network member role 

5% of the 

228. 
estimated target 
population (cascade 
approach) 

n.a. 

222. 

Result 

A higher response rate than recorded in previous exercises The 

distribution of the survey responses is well balanced in terms of 
country and sector of activity in comparison with the target sample. 
One notable exception is France (no responses received) 

Response rates from both coordinator (47%) and partner groups (48%) 

are roughly equal  

Distribution of the survey population is very similar to the target sample 
Over two thirds of respondents located in EU15 Members States, but 
sufficient respondents located in EU13 MS to analyse questions 
based on country groupings 

Number of respondents from COSME-participating third countries too 

low to make meaningful analysis. 

230. 

35% 

Funded versus 

The response rate for funded organisations was higher than that for 

voluntary ESCPs 

voluntary organisations 

232. 

41% 

Country grouping  

235. 

28% 

Country grouping  

238. 

15% 

Country grouping 

High level of alignment between the country groups split between the 
target sample and respondent groups, for those for which this 
information was available. 

Distribution of responses (among those respondents for which the 

geographical location is known) is very similar to that of the target 
sample 

Distribution of responses (among those respondents for which the 

geographical location is known) is very similar to that of the target 
sample 

223. 
SO1 - 
LGF beneficiaries 

225. 
SO2 - 
Network member 
organisations 

SO2 - 

227. 
Network client 
SMEs 

SO3 – 

229. 
Clusters Go 
International 

231. 
Tourism 

SO3 –

233. 
SO4 - 
New entrepreneurs 
234. 
236. 
SO4 - 
Host entrepreneurs 
237. 

The qualitative targeted consultations 

As  shown  in  Section  0,  the  qualitative  consultations  gave  different  results.  While  three  out  of  the  four 
surveys related to SO1 reached a reasonable level of response rate, the same can be said only for one out of 
four surveys related to SO3. 

55 

 
 
 
 
 
 
 
 
The  qualitative  surveys  for  which  the  number  of  returned  questionnaires  was  below  expectation  are  the 
ones targeting the EFG beneficiary SMEs, and the surveys related to SME policy actions (one out of two), 
the  sectoral  competitiveness  actions,  and  the  e-skills  actions.  A  factor  playing  a  role  for  the  EFG 
beneficiaries  was  the  cascade  approach  used;  for  the  sectoral  competitiveness  one,  the  wide  variety  of 
sectors  to  be  covered  was  a  hindering  factor,  as  well  as  the  related  difficulty  to  identify  the  relevant 
stakeholders to include, even with the support from the EC. 

Overall, the quality of the responses was also often below the expected level; the full exploitation of this 
evaluation  tool  was  hindered  by  the  impossibility  for  the  evaluation  team  to  set  up  a  direct  follow-up 
interaction  with  the  respondents.  These  consultations  did  not  have  the  intention  to  reach  ‘statistical 
representativeness’. 

The public consultation 

Public  consultations  present  limits  for  use  in  the  context  of  an  evaluation  due  to  the  impossibility  of 
controlling and checking the ‘statistical representativeness’ of the information provided. For  example, the 
profile  of  the  respondents  shown  in  Section  0  clearly  indicates  a  strong  representation  of  stakeholders 
involved in the financial instruments and respondents involved in a mix of other COSME actions. A use of 
these  data  in  a  statistical  manner  would  therefore  have  been  inappropriate.  Instead,  we  considered  the 
responses as well as the opinion papers submitted as illustrations of a viewpoint, in contrast to our original 
intentions,  used  the  data  exclusively  to  corroborate  findings  deriving  from  various  other  data  collection 
tools used. 

SUMMARY OF THE RESULTS  

Relevance 

Quantitative consultations 

Looking at the financial instruments, it appears that they matched the beneficiaries’ needs. More than half 
of the respondents indicated that they preferred the option that included the EU-COSME guarantee, though 
other sources of finance were available to them that would cover all or part of their required amount. The 
survey  shows  that  the  EEN  member’s  view  on  the  extent  to  which  Network  services  help  SMEs  in 
addressing  their  needs  is  in  general  quite  positive.  There  is  a  good  match  between  the  type  of  services 
financed  by  the  Network  and  the  demand  of  companies.  When  it  comes  to  tourism,  respondents  were 
positive, with each of 12 needs rated as being addressed ‘to a great extent’ or ‘to some extent’ by at least 
two  thirds  of  them.  All  cluster  organisations  responded  positively  to  the  benefits  resulting  from 
collaboration,  calling  for  the  creation  of  stronger  linkages  between  locations  brought  about  by  COSME 
actions.  When  it  comes  to  the  EYE,  the  respondents’  answers  provide  widespread  confirmation  of  the 
existence  of  market  and  systemic  failures  in  the  area  of  barriers  to  entrepreneurship  and  entrepreneurial 
culture were present in their country today. 

Interviews 

Financial intermediaries interviewed mentioned that some financial intermediary needs – and therefore, in 
cascade,  SME  needs  -  are  not  sufficiently  supported  by  the  LGF.  Interviewed  stakeholders  regarded  the 
focus of the EFG instrument as appropriate. Interviewed intermediaries indicated that their  EEN services 
matched the  needs of companies and SMEs in the field very  well. Interviewees indicated the creation of 
knowledge-exchange opportunities is perceived as critical in the field of KETs. They considered there was 
a gap in the support from COSME from that perspective; more support for awareness raising activities was 
seen  as  beneficial.  Stakeholders  interviewed  agreed  that  support  by  the  SME  policy  actions  is  highly 
relevant for the dissemination of the ‘Think Small First’ principle. 

Public consultation 

Networking  actions  rank  as  the  most  relevant  actions  of  the  programme  amongst  the  respondents.  Two 
thirds  of  them  considered  network-building  and  cooperation  as  relevant.  Financial  instruments  come 
second, with on average, a little less than two thirds of the respondents thinking grants and loan portfolio 
guarantees are relevant. 

Effectiveness 

Quantitative consultations 

56 

 
to 

enable 

appears 

focused 

investments 

The  EU-COSME  guarantee 
in  purchases  or 
improvements/developments  among our  survey respondents in areas of capital expenditure, regardless of 
business  size.  Respondents  were  rather  positive  about  the  potential  for  the  financing  to  further  enable 
expansions  in  existing  markets  and  the  development  of  products  or  services  that  are  new  to  the  market. 
Results  from  the  survey  show  that  the  services  related  to  knowledge  enhancement  of  the  EEN  are  rated 
highly by  the  SMEs. Moreover, Network  members stated that their participation in the  Network allowed 
them to build stronger  networks  for their organisation. When it comes to  tourism, respondents’ answers 
suggest  that  COSME-funded  tourism  projects  are  achieving  their  expected  objectives  in  relation  to 
visibility  and  capacity.  Looking  at  cluster  organisations,  over  half  of  the  responding  organisations 
reported  that  they  had,  at  least  to  some  extent,  achieved  (or  were  expected  to  in  the  near  future)  the 
objectives  of  the  actions.  A  majority  of  EYE  beneficiaries,  i.e.  the  New  Entrepreneurs,  reported 
improvements  in  all  of  the  suggested  skill  areas,  and  in  particular  various  personal  and  language  skills. 
Host Entrepreneurs reported widespread benefits for their business activities as a result of hosting a New 
Entrepreneur. 

Interviews 

According to interviewees, the EEN services had a positive indirect effect on business cooperation at the 
regional  level  thanks  to  participation  in  the  information  events.  The  interviewees  emphasised  on  the 
importance of the knowledge- and network-sharing activities among the Network members for the overall 
functioning and quality of the services provided to SMEs. The interviewees considered the e-skills actions 
effective especially in raising awareness on the needed skills and in engaging the relevant stakeholders and 
policy-maker  communities.  Most  national  representatives  interviewed  stated  that  the  SME  performance 
review provides reliable statistical data and analysis on EU SMEs and SME policies. Some interviewees 
mentioned that the SME Envoys network could be more exploited. The interviewees find that the clusters 
and cluster managers are well placed to facilitate the internationalisation of their member SMEs as cluster 
managers  are  acquainted  with  the  technologies  of  the  companies  in  their  network  and  can  promote  the 
“ecosystem” to outside markets. 

Public consultation 

COSME fully responded to its key objective of supporting jobs, growth and investment,  with half of the 
respondents  who  believe  that  the  programme  largely  contributed  to  it.  Additionally,  almost  half  of  the 
respondents  believe  that  the  COSME  programme  supports  the  creation  of  start-ups,  the  growth  of  SMEs 
and business transfer. As much respondents believe that the programme eases access to finance for SMEs 
through loans.  Improved international  visibility and  more  cooperation  with partners  from other countries 
are the two main expectations respondents have regarding COSME 

Efficiency 

Quantitative consultations 

Survey  responses  paint  a  positive  picture  of  the  EU-COSME  guarantee.  The  majority  of  respondents 
reported that they had not experienced additional costs, administrative burden or complexity as a result of 
their EU-COSME guarantee. When it comes to the EEN members, the survey results show that only 9% of 
the Network members state that benefits of the Network do not outweigh the time investment. Regarding 
the tourism actions, respondents were generally positive about both (financial and non-financial) aspects, 
with 85% (122 respondents) either strongly agreeing or agreeing that the Programme provides funding in a 
time-efficient  manner.  Looking  at  cluster  organisations,  over  a  third  of  them  thought  the  reporting  and 
monitoring requirements set by the programme were reasonable. However, many of the organisations felt 
the  funding  requirements  were  not  correctly  estimated  by  the  ‘Clusters  Go  International’  programme. 
Though  New Entrepreneurs do receive programme  funding to cover certain travel, accommodation and 
other costs during their placement, our survey suggests that this is often not sufficient to cover costs during 
their  stay.  Most  respondents  were  broadly  satisfied  across  all  of  the  aspects  covered  by  the  survey, 
including  the  timing  and  administration,  as  well  as  the  support  and  assistance  that  was  provided  by 
intermediary organisations at different stages. 

Interviews 

The  interviews  indicated  that  EASME  is  quite  efficient  in  the  tasks  they  run  (EEN)  thanks  to  the 
experience built up during the CIP programming. The transition to the COSME programme did not have a 
significant impact on their tasks, in which efficiency was preserved. The availability of EASME to respond 
to  questions  from  the  Network  members  was  highly  praised  by  the  interviewees.  CGI  participants 
interviewed felt unrealistic expectations were set by the EC especially for Strand 2 actions. 

57 

 
Public consultation 

According to the respondents, the COSME procedures and administrative documents are fair, transparent 
and  clear.  In  addition,  the  overall  programme  is  considered  as  well  structured.  On  the  other  hand,  the 
programme’s biggest weakness is in its capacity to communicate and reach the potential beneficiaries (less 
than one in five respondents thinks it has the capacity). A couple of questions were targeted at respondents 
from  non-EU  participating  countries.  Overall,  it  appears  that  the  COSME  programme  has  not  been  very 
convincing to those respondents. 

Coherence 

Quantitative consultations 

When it comes to the LGF, it is difficult to make robust conclusions about the complementarity of the EU-
COSME  guarantee  and  other  initiatives,  as  there  is  little  uptake  (and  thus  opinion  on)  other  sources  of 
funding  (both  European  and  national  or  regional)  for  the  same  or  similar  activity  among  beneficiaries. 
Looking  at  the  EEN  and  other  EU  programmes,  survey  results  show  a  high  level  of  complementarity 
between the Network and H2020. At national level, the maturity of the role of the Network members needs 
to grow, while the recognition of the Network by stakeholders in the region needs to be further developed. 
Though at least 50% of COSME tourism actions beneficiaries said they were aware of the campaigns, the 
respondents  indicated  a  generally  low  uptake  of  the  tools.  Survey  respondents  were  rather  more  positive 
when asked to what extent they use the tourism-related networking opportunities offered by the European 
Commission  through  the  COSME  programme.  Regarding  the  cluster  organisations,  the  proportion  of 
respondents  reporting  complementarity  between  Clusters  Go  International  and  other  activities  greatly 
outweighed  those  reporting  any  overlap  with  any  of  these  other  schemes.  When  it  comes  to  the 
Entrepreneurship TA, survey respondents did not point at specific overlaps between individual COSME 
entrepreneurship actions.  

Interviews 

Desk research and interviews showed a high level of synergy with the EEN by allowing SMEs to send out 
questions  from  the  portal  to  the  Network  members.  Interviews  show  that  there  is  a  high  level  of 
complementarity between the Network and H2020. Interviewees considered that there is room for creating 
more  synergies between  the cluster  internationalisation actions and the actions  supporting cluster  value 
chains for S3. A more ‘holistic’ integrated approach to supporting Cluster internationalisation as part of 
the COSME Programme would be beneficial. Interviewees mentioned the need to coordinate the approach 
to cluster internationalisation with the new COSME cluster support actions targeting the strengthening of 
value  chains  for  smart  specialisation.  In  the  context  of  the  Cluster  internationalisation  programme, 
interviewees  indicated  that  there  is  no  clear  coordination  between  EU-level  and  national  level  cluster 
policies. 

Public consultation 

On  average,  two  thirds  of  the  respondents  did  not  know  what  to  answer  regarding  the  coherence  of 
COSME  with  other  EU  programmes.  Nonetheless,  Horizon  2020,  the  European  Fund  for  Strategic 
Investment  (EFSI)  and  the  European  Structural  and  Investment  Funds  (ESIF)  are  the  programmes  with 
which COSME works the most in synergy/is the most complementary. 

European added value 

Quantitative consultations 

Survey  responses  show  that  the  EU-COSME  guarantee  creates  significant  value  for  beneficiaries.  The 
EU-COSME  guarantee  both  provides  funding  for  those  that  had  struggled  to  receive  funding  for  their 
projects from other sources, as well as supporting projects to be conducted at a larger scale than would be 
possible  without  the  funding.  The  survey  results  show  that  responding  SME  clients  held  quite  positive 
views about the importance of the EU level Network services as opposed to only national support. Survey 
respondents  were  very  positive  about  the  value  of  EU  level  support  in  the  field  of  tourism.  Survey 
respondents were also very positive when asked to what extent they agree that the tourism-related actions 
funded by COSME have resulted in specific developments that would not have happened otherwise. Four 
out  of  five  cluster  organisations,  part  of  the  Clusters  Go  International  programme,  strongly  agree  that 
cluster  cooperation  is  essential  to  create  a  more  competitive  European  Union,  while  none  of  the 
organisations  disagreed  with  this  statement.  Host  and  New  Entrepreneurs  who  have  participated  in  the 

58 

 
EYE programme were nearly unanimous in their belief that the EYE scheme should continue in the future.  
Indeed, the vast majority of both groups stated that this was ‘very important’.   

Interviews 

Interviewees  generally  agree  that  there  are  no  private  institutions  nor  national  governments  that  can 
achieve  the  results  that  COSME  (financial  instruments)  has  managed  to  achieve  in  terms  of  leverage 
effects  and  scope  and  risk  profile  of  the  SMEs  involved.  In  the  interviews,  the  intermediaries  clearly 
indicated that for all the ‘structural’ elements of the  Network, the European dimension constitutes a pre-
condition. Many interviewees indicated a shortage of funding in their countries or regions for this type of 
SME support; in this case, the additionality is therefore of a financial kind. 

Public consultation 

Overall,  a  third  of  the  respondents  considers  that  the  overall  value  of  COSME  compared  with  national 
and/or  regional  SMEs  and  entrepreneurship  support  programmes  is  higher.  More  than  half  of  the 
respondents believe the  programme helped financing projects  which otherwise  could  not be supported at 
national and/or regional level. 

59 

 
Annex 3: Methods and analytical models 

60 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Table of Contents 

1 
2 

INTRODUCTION ............................................................................................................................... 66 
THE METHODOLOGICAL FRAMEWORK .................................................................................... 67 

2.1 

2.2 

2.2.1 

2.2.2 

Methodological approach  67 

Overview of the evaluation tools and methods 

Brief description of tools and methods 

Matrix for the triangulation 

72 

2.3 

Robustness of the results  73 

2.3.1 

2.3.2 

2.3.3 

The quantitative targeted consultations 

The qualitative targeted consultations 

The public consultation 

75 

69 

70 

74 

75 

2.4 

Flow of the analysis 75 

3 

DESK RESEARCH ............................................................................................................................. 76 

3.1 

3.2 

3.2.1 

3.2.2 

3.2.3 

3.2.4 

Overview  76 

Details at the Thematic Areas level  78 

SO1 – Access to finance 

SO2 – Access to market 

78 

79 

SO3 – Favourable environment 

80 

SO4 – Entrepreneurship  81 

4 

STATISTICAL DATA ANALYSIS ................................................................................................... 82 

4.1 

4.2 

4.3 

4.4 

4.5 

Sources 

82 

Portfolio analysis  82 

Composition analysis 

83 

Analysis of COSME programme management and proposal data 

84 

Cost-Effectiveness analysis  86 

5 

THE CONSULTATIONS ................................................................................................................... 93 

5.1 

The targeted consultations – quantitative  93 

5.1.1 

5.1.2 

5.1.3 

Overview  93 

Sampling strategy and results 

93 

Robustness of the results  97 

5.2 

The targeted consultations - qualitative 

106 

5.2.1 

5.2.2 

5.2.3 

5.2.4 

Overview  106 

Sampling strategy  106 

Results for SO1 – Access to finance 106 

Results for SO3 – Favourable environment  107 

61 

   
 
 
Interim Evaluation of the COSME Programme - Final Report  

5.3 

The public consultation 

108 

5.3.1 

5.3.2 

5.3.3 

5.3.4 

The structure of the public consultation questionnaire 

108 

The implementation of the public consultation 

109 

Outcomes of the public consultation 

109 

Statistical analysis  110 

6 

INTERVIEWS ................................................................................................................................... 111 

6.1 

6.2 

Overview  111 

Results 

111 

6.2.1 

6.2.2 

6.2.3 

6.2.4 

SO1 – Access to finance 

SO2 – Access to market 

111 

112 

SO3 – Favourable environment 

112 

SO4 – Entrepreneurship  112 

APPENDIX  A

  -  COSME  PROGRAMME  KPIS,  AND  THE  EXTENT  TO  WHICH  THESE  KPIS 

APPENDIX B

APPENDIX C

HAVE BEEN REALIZED ................................................................................................................ 114 
 -  STATISTICAL DATA ANALYSES RESULTS ............................................................ 122 
 - LIST OF DOCUMENTS ANALYSED ........................................................................... 129 
 – LISTS OF INTERVIEWEES .......................................................................................... 139 

APPENDIX D

List of Tables 

Table 1 Evaluation questions ............................................................................................... 68 

Table 2 Tools and methods used in the evaluation .................................................................. 70 

Table 3 Use of the methodological mix per evaluation question ................................................ 72 

Table 4 Literature list at the programme level ........................................................................ 76 

Table 5 Evidence table for the desk research and literature review ........................................... 77 

Table 6 Preliminary list of the available relevant databases ...................................................... 82 

Table 7 Portfolio analysis levels ............................................................................................ 83 

Table 8 Overview of available data on beneficiaries ................................................................ 84 

Table 9 Overview of available data on intermediaries .............................................................. 84 

Table 10 Calls on which time to grant analysis is based ........................................................... 84 

Table 11 Time to grant by thematic area and action line.......................................................... 86 

Table 12  Cost-effectiveness analysis .................................................................................... 88 

Table 13 Overview – Quantitative surveys ............................................................................. 93 

Table 14 Overview of the participation in the qualitative surveys ............................................ 106 

Table 15 Overview of status of qualitative surveys................................................................ 107 

62 

   
 
 
 
 
Interim Evaluation of the COSME Programme - Final Report  

Table 16 Overview of responses to SO3 short survey consultations ......................................... 107 

Table 17 Overview of the open public consultation process .................................................... 109 

Table 18 Number of respondents per type of financial support ............................................... 110 

Table 19 Overview of the interviews conducted, by SO .......................................................... 111 

Table 20 SO3 – Performed interviews ................................................................................. 112 

Table 21  Legend to Table 22 below .................................................................................... 115 

Table 22  Main activities, KPIs, and the extent to which these KPIs have been realised ............. 115 

Table 23 Country penetration rates of LGF guarantee 2014-2016 ........................................... 122 

Table 24 Geographical distribution - shares by country aggregation of number of beneficiaries for 

2014-2016 ............................................................................................................... 122 

Table 25 Geographical distribution – estimated number of final recipients by country in EU for 

2014-2016 ............................................................................................................... 123 

Table 26 Geographical distribution – weighted shares of participants by country in EU for 2014-

2016 ....................................................................................................................... 124 

Table 27 Geographical distribution - shares by country aggregation of cumulative budget for 2014-

2016 ....................................................................................................................... 125 

Table 28 Geographical distribution (money allocated to final beneficiaries in EUR k for 2014-2016)

 .............................................................................................................................. 125 

Table 29 Sectoral involvement (estimated shares of cumulative budget 2014-2016) ................. 126 

Table 30 Sectoral involvement (shares of final beneficiaries 2014-2016) ................................. 127 

Table 31 Sectoral involvement (budgets in EUR m) ............................................................... 128 

Table 32 Sectoral involvement (number of final beneficiaries) ................................................ 129 

Table 33 List of documents analysed SO1 ............................................................................ 130 

Table 34 List of background documents SO2 ........................................................................ 133 

Table 35 List of background documents SO3 ........................................................................ 135 

Table 36 List of documents analysed SO4 ............................................................................ 137 

Table 37 Interviews performed SO1 .................................................................................... 139 

Table 38 Interviews performed SO2 .................................................................................... 140 

Table 39 Interviews performed SO3 .................................................................................... 141 

Table 40  Interviews not performed SO3 ............................................................................. 142 

Table 41 Interviews performed SO4 .................................................................................... 142 

Table of Figures 

63 

   
 
 
 
 
Interim Evaluation of the COSME Programme - Final Report  

Figure 1 Evaluation criteria and intervention logic in an interim/mid-term evaluation .................. 67 

Figure 2 Intervention logic of the COSME programme ............................................................. 69 

Figure 3 Flow of the analysis ................................................................................................ 76 

Figure 4  LGF beneficiaries: comparison of the country distribution of the whole population, target 

sample and survey population ...................................................................................... 98 

Figure 5 LGF beneficiaries: comparison of the sector distribution of the whole population, target 

sample and survey population ...................................................................................... 98 

Figure 6  Network member survey target sample and respondents, by county ........................... 99 

Figure 7  Network member survey target sample and respondents, by role ............................. 100 

Figure 8  Network client SME company survey respondents, by country group ......................... 102 

Figure 9  Distribution of population in sample and responses between type of cluster organisation

 .............................................................................................................................. 102 

Figure 10  Profile of target sample and SO3 tourism survey respondents, by country group ....... 103 

Figure 11  Distribution of population in sample and responses by types of countries ................. 104 

Figure 12  Distribution of population in sample and responses by types of countries ................. 105 

Figure 13 Public consultation - profile of the participants ....................................................... 110 

List of abbreviations 

Abbreviation 
CEA 
CIP 
COSME 

DG CONNECT 
DG GROW 
DG REGIO 
DG RTD 
DG TRADE 
DGT 
DG EAC 
EASME 
EC 
EEN 
EFG 
EIB 
EIF 
EIP 
ESCPs 
EU 
EYE 
FWC 
HE 
ICT 
IPR 
KETS 
LGF 
NACE 
NCP 
PES 
RPAS 
SBA 
SMEs 
SO 

Full name 
Cost-Effectiveness Analysis 
Competitiveness and Innovation Framework Programme 
Programme for the Competitiveness of Enterprises and Small- and Medium-sized 
Enterprises 
Directorate-General for Communications Networks, Content and Technology 
Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs 
Directorate-General for Regional and Urban Policy 
Directorate-General for Research and Innovation 
Directorate-General for Trade 
Directorate-General for Translation 
Directorate-General for Education, Youth, Sport and Culture 
Executive Agency for SMEs 
European Commission 
European Enterprise Network 
Equity Facility for Growth 
European Investment Bank 
European Investment Fund 
European Innovation Partnerships 
European Strategic Cluster Partnerships 
European Union 
Erasmus for Young Entrepreneurs 
Framework Contract 
Host Entrepreneur 
Information and Communication Technology 
Intellectual Property Rights 
Key-Enabling Technologies 
Loan Guarantee Facility 
Statistical Classification of Economic Activities in the European Community 
National Contact Point 
Performance Enhancement System  
Remotely Piloted Aircraft Systems  
Small Business Act 
Small- and Medium-sized Enterprises 
Specific Objective 

64 

   
 
 
 
Interim Evaluation of the COSME Programme - Final Report  

SPR 
UEAPME 

SME Performance Review 
European Association of Craft, Small- and Medium-sized Enterprises 

65 

   
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

INTRODUCTION 

This annex is structured as follows: 

  The methodological framework used for the evaluation is presented in Section 1 

  We then describe the approach taken and provide information on the results at SO level 

for the methods used for the evaluation (including the case studies), i.e. the desk research 
(Section 2), the statistical data analysis (Section 3), the targeted and public consultations 
(Section 4) and the interviews (Section 5). 

There are four appendices to the report, providing additional data (Appendix A and Appendix 
B),  lists  of  documents  analysed  (Appendix  C)  and  interviewees  (Appendix  D),  and  the 
targeted (quantitative) and public consultation questionnaires). 

66 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

1.  THE METHODOLOGICAL FRAMEWORK 

In this chapter, we give an overview of our approach taken to the evaluation. Section 1.1 
presents  the  key  concepts  that  guided  our  evaluation,  including  the  programme’s 
intervention logic and the evaluation questions.  

Section  1.2  summarises  the  evaluation  tools  and  methods  we  used,  the  triangulation 
matrix we designed and the flow of analysis we followed. 

The  objective  of  the  study  was  to  evaluate  the  relevance,  effectiveness,  coherence, 
efficiency  and  EU  added  value  of  the  programme  activities  funded  in  the  2014-2016 
period.  The  study  findings  should  contribute  to  the  decision-making  on  the  renewal, 
modification or suspension of the programme actions and measures, to set the basis for 
the  next  evaluation  of  the  COSME  programme  until  2020  and  contribute  to  the 
preparation of the future programme in the next Multiannual Financial Framework (MFF), 
and to feed into the mid-term evaluation of EASME. 

The  scope  of  this  evaluation  was  the  COSME  activities  conducted  in  the  period  2014-
2016.  

1.1.  Methodological approach 

The Terms of Reference (ToR) indicated the evaluation criteria of relevance, efficiency, 
effectiveness, coherence and  EU added  value as overall framework  for the  study. These 
evaluation  criteria  assess  the  relationships  between  the  various  components  of  the 
intervention logic (Figure 5). 

Figure 5 Evaluation criteria and intervention logic in an interim/mid-term evaluation 

Source: Technopolis 

During the inception phase, the scope for the analyses against these evaluation criteria 
was defined as follows: 

  Assessment of the relevance, coherence, and efficiency in implementation criteria 

covering all actions  

67 

ObjectivesResourcesActivitiesOutputsShort-termOutcomesIntermediateimpactsLong term ImpactsEfficiencyEffectivenessNeeds, problems, issuesRelevanceEU policyOther initiativesCoherenceEU added valueExternal environmentStakeholders reached 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

  Assessment against the effectiveness criterion focused on the Key Actions, 

complemented by four case studies  

  The European added value and the efficiency / cost-effectiveness criteria remain 

covered through the analysis of the Key Actions only 

The Key Actions included: the financial instruments (LGF and EFG), the Enterprise Europe 
Network  services,  the  SME  Policy  actions,  the  Cluster  Internationalisation  programme, 
and the Erasmus for Young Entrepreneurs programme.  

Actions covered through case studies are: the Vulcanus scheme in the  EU-Japan Centre 
action;  the  e-skills/digitisation  actions,  the  Tourism  actions  (grants),  and  the  Women 
Entrepreneurship action.  

The  supporting  study  was  performed  on  the  basis  of  six  leading  evaluation  questions 
and fifteen sub-questions for this evaluation.  

Table 11 Evaluation questions 

Key question 
1) Implementation - State of play  Q1.1) How has the programme been implemented and how has it evolved 

Evaluation question 

2) RELEVANCE: Do the objectives 
correspond to the current needs? 

3) EFFECTIVENESS: How effective 
is the programme in achieving its 
objectives? 

4) EFFICIENCY: To what extent 
were the effects (benefits) 
achieved at a reasonable cost? 

5) COHERENCE: To what extent 
are the individual actions financed 
by COSME internally coherent and 
to what extent is COSME 
externally coherent with other EU 
and MS programmes/policies with 
similar objectives? 
6) EU ADDED VALUE: What is the 
EU value added? 

Source: Technopolis 

since the start in 2014? 
Q2.1) To what extent are COSME’s initial objectives still pertinent to the needs, 
problems and issues it was designed to address? 
Q2.2) How relevant is the programme to the stakeholders (public and private) 
and the EU citizen? 
Q2.3) How has the programme adapted to the political priorities, for instance 
to the new Single Market Strategy and its particular objective to support Start-
ups and Scaling-ups? 
Q3.1) To what extent is the COSME programme progressing towards achieving 
its specific objectives? 
Q3.2) What are the factors driving or hindering progress and how do they link 
with different actions of the programme? 
Q3.3) To what extent is the Programme progressing towards contributing to 
the 5 Union priorities for smart, sustainable and inclusive growth (incl. climate 
objectives and gender mainstreaming)? 
Q4.1) What are the regulatory/administrative costs and the benefits for the 
different stakeholders? 
Q4.2) Are the costs (direct and indirect) generated by the programme 
proportionate to the benefits generated? 
Q4.3) How efficient is the implementation structure and governance of the 
programme? 
Q5.1) Are the different actions of COSME implemented coherently between 
them? 
Q5.2) How are the different COSME actions overlapping/synergizing with the 
activities of ESI funds, H2020, EFSI, EASI and other EU programmes? 
Q5.3) To what extent has the coherence been maximised in view of National 
programmes?  

Q6.1) What is the added value of the programme and actions being conducted 
at the EU level? 

Q6.2) To what extent and why do the issues addressed by the programme 
continue to require action at EU level? 

Time  is needed  for  effects  such  as  mid-term  results  and long-term  impacts  to  occur.  In 
the  case  of  interim  evaluations  such  as  the  current  one  focusing  on  the  COSME 
programme, most of the expected mid-term results are not yet be measurable. The focus 
is  therefore  on  the  assessment  whether  the  programme  succeeded  in  setting  the 
conditions  that  can  be  expected  to  lead  to  these  expected  results.  This  implies  an 
emphasis  in  this  evaluation  on  the  adequacy  and  quality  of  the  programme  design,  i.e. 

68 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

the  formulated  objectives,  the  invested  resources,  and  the  activities  implemented  –  in 
other words, the relevance and coherence criteria.  

In  terms  of  effectiveness,  the  assessment  focused  on  the  attainment  of  the  expected 
outputs and short-term outcomes, and the enabling factors or barriers for the attainment 
of  the  outcomes  and  expected  results,  such  as  the  capacity  of  the  programme  to  reach 
and involve the stakeholders needed. 

The intervention logic was a critical component of our methodological framework. The 
consistent  use  throughout  our  analyses  of  the  categories  and  types  of  effects  that  were 
defined  in  this  intervention  logic  allowed  us  to  structure  our  findings  and  draw 
conclusions at the programme level, going beyond a mere aggregation of the findings at 
the SO level. Figure 6, below, sets out the intervention logic of the COSME programme 
and its expected effects. 

Figure 6 Intervention logic of the COSME programme 

Source: Technopolis 

1.2.  Overview of the evaluation tools and methods 

The consultant used a mix of qualitative and quantitative methods for the evidence 
collection  and  analysis.  We  briefly  describe  our  approach  for  these  evaluation  tools  and 
methods in Section 1.2.1, below. A full description of each tool and method is provided in 

69 

Themes & action lines•Access to finance•Business management capacity•DigitisationPolicy instruments•Financial instruments•Information instruments•Framework conditions for the Single Market•Entrepreneurship•InternationalisationInputKnowledge & learning•Knowledge networks & platforms•Information & awareness platforms & tools•Outreach activities•Business management coaching/trainingCompetitiveness and Growth•Risk capital & loans•Market intelligence platforms & tools•Analytical tools (egself-assessment tools)•Business networking platforms & eventsPolicy•Platforms/tools for strategic intelligence•Networks & platforms for policy coordination•Networks & platforms for policy knowledgeOutputsStrengthened global competitivenessJobs & GrowthBusiness creation & growthSustainable & inclusive  GrowthSkills•Improved technical & managerial skills•Enhanced business & innovation management•Strengthened entrepreneurial skillsKnowledge•Knowledge spillovers•Improved access to knowledge & information•Enhanced market knowledgeFinance•Easier access to finance & capital•Start-ups created or supportedNetworks•Cross-EU industry cooperation & relationships•International industry cooperation•Cross-sectoralpartnerships•Public-private collaborationsInnovation•New or modified business models•New or modified products, processes, servicesFramework conditions•Coordinated IPR, standards & regulationsPolicy•Enhanced policy knowledge•Improved policy coordination•Enhanced strategic intelligence•Better evidence-based policy-making•Removal existing administrative barriers•Simplified regulatory frameworkOutcomesImpactsCapacity enhancement•Enhanced culture of entrepreneurship•Greater investment in human capitalConditions for growth•Innovation in policy•Better market conditions•Strengthened Single Market•Increase in private investment,  including in R&I•Better integration of capital markets•Pan-European finance marketInnovation•Diffusion of innovationScaling up•Improved access to finance•Improved access to market•Strengthened internationalisation•SMEs integrated in global value chainsResults•Enhanced business management capacity•Enhanced innovation management capacity•Enhanced internationalisationcapacity•Responsible innovation•Sectoral competitiveness•SME policy•Knowledge platforms & tools•Human capital measures•Knowledge networks & schemes 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

the next chapters. In Section 1.2.2 we present the matrix that constituted the framework 
for the triangulation of the evaluation findings. Section 1.3 sets out the flow of analysis in 
the evaluation and its workplan. 

1.2.1. 

Brief description of tools and methods 

Table 12, below, presents the tools and methods used in this evaluation. 

Table 12 Tools and methods used in the evaluation 

Methodological approach 

Tools & methods 

Qualitative methods 

Desk research 

Interviews 

Case studies 

Workshop 

Qualitative targeted consultation 

Targeted consultation  

Quantitative methods 

Public consultation 

Statistical analyses of secondary data  

Number 

260+ 

121 

4 

1 

7 

7 

1 

5 

Qualitative methods 

The  desk  research  encompassed  a  broad  range  of  sources,  ranging  from  programme 
and policy documents to market studies, impact assessments and preceding and/or other 
evaluations.  The  list  has  been  in  constant  expansion  throughout  the  evaluation.  The 
review of the literature and desk research was an activity that took place throughout the 
entire lifespan of the evaluation. Nevertheless, it was concentrated especially in the first 
stages  of  the  evaluation,  enabling  the  study  team  to  conduct  the  interviews  with  the 
stakeholders in an informed manner.  

A  major  purpose  of  the  high-level  interviews  was  to  collect  background  and  context 
information on facilitators and barriers for the attainment of the expected effects as well 
as  the  eventual  unintended  effects.  We  also  conducted  some  interviews  at  the  overall 
programme  level.  These  interviews  with  EU  officials  took  place  in  the  beginning  of  the 
study to reach an improved understanding of the overall context to the programme and 
the processes for the programme design. 

In this evaluation, we conducted four case studies covering three SOs: SO2 – Access to 
market (the trainee programme “Vulcanus in Japan”, run by the EU-Japan Centre), SO3 – 
Favourable  environment  (the  Tourism  actions  under  the  programme  and  the  E-skills  for 
competitiveness  and  innovation  actions),  and  SO4  –  Entrepreneurship  (the  Women 
Entrepreneurship  action).  Each  case  study  covered  the  overview  of  the  programme’s 
progress  in  attaining  the  objectives  set  out  in  the  intervention  logic,  and  to  the  extent 
possible, an appraisal of their progress towards attaining their expected results. Another 
important focus of the case study was the analysis of the relevance and coherence of the 
COSME actions. The results fed into the wider analysis of SO2, SO3, and SO4, as well as 
of the COSME programme overall. 

A day-long workshop was organised on May 31, 2017, involving representatives of the 
Member  States  and  the  COSME  Third  Countries  as  well  as  Key  Stakeholders.  The 
objective  of  the  workshop  was  two-fold:  presenting  the  then-current  results  to  the 

70 

 
 
  
  
  
 
Interim Evaluation of the COSME Programme - Methodology Report  

stakeholders as well as receiving feedback from them, especially regarding the relevance, 
coherence and European added value of the COSME programme.  

The  first  half  of  the  workshop  was  dedicated  to  the  presentation  of  the  COSME  interim 
evaluation  (the  objectives  and  the  methodology)  and  the  first  findings  of  the  team  for 
each  SO,  followed  by  a  Q&A  session.  During  the  second  half,  participants  split into  four 
groups.  Positive and negative aspects  of the  COSME programme were discussed as well 
as  the  ideal  design  of  an  SME  support  programme.  A  plenary  session  concluded  the 
workshop where the outcomes of the breakout sessions were discussed. 

On  request  of  the  EC,  this  evaluation  included  the  conduct  of  ‘qualitative’  targeted 
consultations, replacing interviews. This type of surveys will be implemented where the 
population is  lower  than  100;  the  small  numbers  therefore  did  not  allow  for  meaningful 
statistical  analysis.  The  questionnaires  mostly  include  open-ended  questions,  geared  at 
collecting qualitative information. 

Quantitative methods 

The  ‘classical’  targeted  consultations  were  implemented  in  those  cases  where  the 
population  was  higher  than  100  and  would  therefore  allow  for  meaningful  and  robust 
statistical  data  analyses.  The  questionnaires  mostly  include  close-ended  questions  and 
only  a  few  open-ended  questions.  The  responses  were  analysed  through  descriptive 
statistics, in several cases also included cross-analyses. 

The public consultation was a 12-week internet-based questionnaire. The objective of 
the  public  consultation  was  to  cover  all  potential  relevant  stakeholders,  in  particular 
those  that  were  not  targeted  through  the  targeted  consultations  and interviews,  and  to 
collect  feedbacks  from  them  on  the  COSME  programme  and  its  objectives.  More 
specifically, we received input from: 

  Business and trade associations 

  Citizens of the EU 

  Non-profit organisations 

  Financial intermediaries (private equity, business angels, venture capital funds, 

banks, promotional banks, guarantee institutions) 

 

Intermediary structures such as incubators/clusters/innovation & growth agencies 

  Public authorities (member states, European agencies) 

The  questionnaire  for  the  public  consultation  was  quite  straightforward  as  it  aimed  at 
reaching  all  types  of  stakeholders.  Given  that  the  targeted  consultations  focused 
exclusively  on  the  specific  objectives,  it  was  agreed  to  use  the  public  consultation  to 
collect data that would provide information on the programme as a whole. 

In total four types of analyses were conducted on secondary data:  

  The portfolio analysis entailed the analysis of the COSME budget distribution over 

the different structural components of the programme, ie the Strategic Objectives and 
their operational (sub)objectives, the policy mix used in the programme (ie the policy 
instruments) and the modes of implementation. We covered both the planned budget 
distribution (based on the Work Programmes) and the actual spending. 

71 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

  The composition analysis focused on the stakeholders that benefit of the COSME 

programme funding. The outcomes of this analysis informed especially the analysis of 
effectiveness, in terms of the capacity of the programme to attract the type of 
stakeholders needed for the attainment of the desired effects and the analysis of 
cost-effectiveness. Dimensions for analysis were the geographical location of the 
beneficiaries, the type of stakeholders involved in the actions funded, such as SMEs, 
professional associations, intermediaries, national/regional administration officials, 
and consultancies providing services to the EC, and the sector of activity of the SMEs 
and other private sector stakeholders involved.  

  The time-to-grant  analysis, based  on EASME management data, and the  success 
rate  analysis  of  the  open  calls  for  grants,  based  on  the  EASME  proposal  database, 
were  conducted  to  feed  into  the  assessment  of  the  COSME  efficiency  in  the 
implementation of the programme. 

  The  cost-effectiveness  analysis  (CEA)  entailed  quantifying  the  not-monetised 
benefits  that  would  be  generated  by  one  Euro  of  total  costs.  Typically,  this  entails 
defining  the  benefit-cost  ratio,  i.e.  dividing  the  benefits  by  costs.  CEA  is  normally 
used  to  identify  the  “value  for  money”  of  programmes.  The  CEA  focused  on  SO 
actions  that  have  similar  objectives,  rather  than  on  the  SOs  in  general.  In  addition, 
for the CEA to provide a meaningful result, a measure to identify ‘good performance’ 
was  needed;  typically,  this  is  obtained  through  comparison.  Our  approach  was  to 
assess the cost-effectiveness  of the SO Actions in such a way that the outcomes for 
the  different  SOs  can  be  compared  to  each  other.  This  implied  a  focus  of  the 
‘effectiveness’ factor on the only comparable outcome indicator possible, the number 
of SMEs and individuals supported.  

1.2.2. 

Matrix for the triangulation 

For  each  evaluation  question,  the  analysis  was  based  on  multiple  sources  and  methods 
(Table 13). This allowed for a proper triangulation of the evidence collected, ensuring the 
robustness of the findings. It also helped mitigating the risk of lack in data availability. 

Table 13 Use of the methodological mix per evaluation question 

Qualitative 

Quantitative  

e
h
t

i

g
n
d
u
l
c
n
i
(

i

s
w
e
v
r
e
t
n
I

)
s
y
e
v
r
u
s

”
e
v
i
t
a
t
i
l

a
u
q
“

h
c
r
a
e
s
e
r

k
s
e
D

i

s
e
d
u
t
s

e
s
a
C

s
p
o
h
s
k
r
o
W

/

s
i
s
y
a
n
a

l

a
t
a
d

y
r
a
d
n
o
c
e
S

X 

X 

X 

X 

1) Implementation - State of play  

Q1.1) How has the programme been 
implemented and how has it evolved since the 
start in 2014? 

s
i
s
y
a
n
a

l

o

i
l

o
f
t
r
o
p
&
n
o
i
t
i
s
o
p
m
o
C

s
i
s
y
a
n
a

l

a
t
a
d

y
r
a
d
n
o
c
e
s

r
e
h
t
O

s
i
s
y
a
n
a

l

s
s
e
n
e
v
i
t
c
e
f
f
e
-
t
s
o
C

”
e
v
i
t
a
t
i
t
n
a
u
q
“
(

a
t
a
d

n
o
i
t
a
t
l
u
s
n
o
c

d
e
t
e
g
r
a
t

s
i
s
y
a
n
a

l

l

a
c
i
t
s
i
t
a
t
S

n
o
i
t
a
t
l
u
s
n
o
c

c
i
l

b
u
P

)
y
e
v
r
u
s

2) RELEVANCE: Do the objectives correspond to the current needs?  

Q2.1) To what extent are COSME’s initial 

X 

X 

X 

X 

X 

X 

72 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Qualitative 

Quantitative  

objectives still pertinent … 

Q2.2) How relevant is the programme to the 
stakeholders (public and private) and the EU 
citizen? 
Q2.3) How has the programme adapted to the 
political priorities … 

X 

X 

X 

X 

X 

X 

X 

3) EFFECTIVENESS: How effective is the programme in achieving its objectives?  

X 

X 

X 

X 

X 

Q3.1) To what extent is the COSME programme 
progressing towards achieving its specific 
objectives? 
Q3.2) What are the factors driving or hindering 
progress and how do they link with different 
actions of the programme? 
Q3.3) To what extent is the Programme 
progressing towards contributing to the 5 Union 
priorities …. 
4) EFFICIENCY: To what extent were the effects (benefits) achieved at a reasonable cost? 
Q4.1) What are the regulatory/administrative 
costs and the benefits for the different 
stakeholders? 
Q4.2) Are the costs (direct and indirect) 
generated by the programme proportionate to 
the benefits generated? 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Q4.3) How efficient is the implementation 
structure and governance of the programme? 

X 

X 

X 

X 

X 

X 

5) COHERENCE: To what extent are the individual actions financed by COSME internally coherent and to what 
extent is COSME externally coherent with other EU and MS programmes/policies with similar objectives? 

Q5.1) Are the different actions of COSME 
implemented coherently between them? 

X 

X 

X 

X 

Q5.2) How are the different COSME actions 
overlapping/synergizing with the activities of ESI 
funds, H2020, EFSI, EASI and other EU 
programmes? 

X 

X 

Q5.3) To what extent has the coherence been 
maximised in view of National programmes?  

X 

X 

6) EU ADDED VALUE: What is the EU value added? 

Q6.1) What is the added value of the programme 
and actions being conducted at the EU level? 

X 

X 

Q6.2) To what extent and why do the issues 
addressed by the programme continue to require 
action at EU level? 
Source: Technopolis 

X 

X 

1.3.  Robustness of the results 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

A key method to ensure robustness of the analysis in an evaluation is the triangulation of 
the  findings,  i.e.  the  validation  of  information  collected  through  one  method  by 
comparing it with the information collected through another method.  

In  this  section,  we  discuss  the  measures  of  triangulation  and  the  findings  of  the 
robustness checks related to the consultations that were implemented in this study. 

  Triangulation is especially important for the validation of findings that derive from the 
statistical  analysis  of  survey  responses,  due  to  the  potential  for  statistical  bias  (see 
Section 2.3.1, below). 

  However,  robustness  checks  and  quality  assurance  measures  should  be  applied  also 
for the other consultation methods (see Section 2.3.2 and Section 2.3.3, below). 

73 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

More detailed information is provided also in the sections for the individual actions in the 
sections below. 

1.3.1. 

The quantitative targeted consultations 

It is well known that surveys may suffer from a series of statistical biases that make the 
treatment of responses quite delicate for statistical purposes. These include: 

  Sampling  bias  –  the  distortion  arising  from  the  non-randomness  of  the  sample 
extraction, or the patent under-representation of relevant geographic areas, sectors, 
etc. 

  Non-response  bias  –  the  distortion  arising  from  the  possibility  that  the  non-

respondents share a characteristic that is important but will be missed in the results 

  Response  bias  –  the  answer  distortion  arising  from  the  way  the  questionnaire  is 
structured,  or  the  perceived  expectations  of  the  sponsor/surveyor,  or  laziness 
(primacy, extreme responding, fatigue, etc.) 

Some  of  these standard  statistical  problems  (sampling  bias,  data  errors)  were  present 
also in the surveys conducted in this study. 

However, a part of the more than satisfactory response rates reached, the profile of the 
responding organisations was in close to all surveys very similar to the target sample, in 
general guaranteeing statistical representativeness.  

A first exception to the rule was the survey of the Network client SMEs. This survey was 
managed  through  a  cascade  approach  and  reached  less  responses  than  expected.  In 
addition,  because  of  the  cascade  approach and  therefore  the  unknown  size  of  the 
population,  the  study  team  could  not  check  the  representativeness  of  the  responses. In 
the main report for this study (section 2.3.4),  we emphasise the negative consequences 
of this approach for the statistical value of survey data in the context of evaluations and 
endorse the recommendation made by the CIP and EIP final evaluation teams to ensure 
the possibility for evaluation teams directly to contact beneficiary SMEs. 

A  second  exception  is  the  survey  related  to  the  Clusters  Go  International  programme 
where  more  funded  than  non-funded  ESCPs  responded  to  the  survey,  compared  to  the 
target  sample.  In  this  case,  however,  this  does  not  imply  forcefully  a  lack  of 
representativeness for the funded ESCPs. 

Surveys to stakeholders involved in the programme are often considered to be unreliable 
sources for information due to the expected bias in the responses by beneficiaries in the 
programme.  We  took  this  into  account  by  analysing  the  data  in  terms  of  distribution  of 
the responses in the different groupings along a 5-point Likert scale and by considering 
especially the ‘extremes’ in these distributions. 

Most important, input provided through the surveys was always corroborated through the 
other methods used 

1.3.2. 

The qualitative targeted consultations 

The  qualitative  consultations  gave  different  results.  While  three  out  of  the  four  surveys 
related to SO1 reached a reasonable level of response rate, the same can be said only for 
one out of four surveys related to SO3. 

74 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

The  qualitative  surveys  for  which  the  number  of  returned  questionnaires  was  below 
expectation are the ones targeting the EFG beneficiary SMEs, and the surveys related to 
SME policy actions (one out of two), the sectoral competitiveness actions, and the e-skills 
actions. A factor playing a role for the EFG beneficiaries was the cascade approach used; 
for  the  sectoral  competitiveness  one,  the  wide  variety  of  sectors  to  be  covered  was  a 
hindering  factor,  as  well  as  the  related  difficulty  to  identify  the  relevant  stakeholders  to 
include, even with the support from the EC. 

Overall,  the  quality  of  the  responses  was  also  often  below  the  expected  level;  the  full 
exploitation  of  this  evaluation  tool  was  hindered  by  the  impossibility  for  the  evaluation 
team to set up a direct follow-up interaction with the respondents. 

These consultations did not have the intention to reach ‘statistical representativeness’, so 
there  is  no  reason  to  doubt  the  value  of  the  input  provided.  However,  the  efficiency  of 
this type of exercise is highly questionable. 

1.3.3. 

The public consultation 

Public  consultations  present  limits  for  use  in  the  context  of  an  evaluation  due  to  the 
impossibility  of  controlling  and  checking  the  ‘statistical  representativeness’  of  the 
information  provided.  In  the  case  of  this  evaluation,  for  example,  the  profile  of  the 
respondents  clearly  indicates  a  strong  representation  of  stakeholders  involved  in  the 
financial instruments and respondents involved in a mix of other COSME actions. A use of 
these data in a statistical manner would therefore have been inappropriate. Instead, we 
considered  the  responses  as  well  as  the  opinion  papers  submitted  as  illustrations  of  a 
viewpoint,  and  used  the  data  exclusively  to  corroborate  findings  deriving  from  various 
other data collection tools used. 

1.4.  Flow of the analysis  

The  evaluation  was  implemented  in  three  phases:  the  inception,  data  collection  and 
analysis,  and  reporting  phases,  whereby  the  latter  two  phases  took  place  in  two  steps 
each (Figure 7). 

In  the  data  collection  and  analysis  phase  we  focused  our  activities  on  setting  out  the 
‘state-of-play’  of  the  COSME  implementation  and  the  conduct  of  the  analyses  (desk 
research  and  interviews)  focusing  on  the  relevance  and  coherence  criteria.  This  allowed 
us to report on our preliminary findings for these criteria in the Interim report (D3). In 
the  interim  report  we  also  reported  on  the  progress  in  the  targeted  and  public 
consultations.  

These  preliminary  findings  were  further  validated  in  the  next  step,  in  particular  during 
the  workshop  with  the  COSME  committee  members.  The  collection  of  evidence  through 
the other methods and its triangulation led to the formulation of findings at the thematic 
area  level.  These  findings  were  reported  internally,  allowing  for  an  aggregated  analysis 
and  the  drawing  of  conclusions  and  drafting  of  recommendations  at  the  COSME 
programme  level,  which  were  then  reported  in  the  draft  and  final  version  of  the  Final 
report. 

75 

 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Figure 7 Flow of the analysis 

2.  DESK RESEARCH 

In  this  chapter  we  present  the  desk  research  strategy  followed  for  the  evaluation.  We 
present an overview of this strategy in Section 2.1 while we detail it at thematic level in 
Section 2.2. 

2.1.  Overview 

In Appendix B we provide the full list of all documents consulted per SO.  

Table  14,  below,  lists  the  documents  and  studies  that  were  of  relevance  at  the  overall 
programme level.  

Table 14 Literature list at the programme level 

Type of document 
COSME programme 
documentation 

Examples of relevant documents 
EU (2013), Regulation (EU) No 1287/2013 of the European Parliament and of the Council 
establishing a Programme for the Competitiveness of Enterprises and small and medium-
sized enterprises (COSME) (2014 - 2020) and repealing Decision No 1639/2006/EC.  
Ex-ante Preparatory documents for COSME, including public consultation: Final report, 
annex 1, 1a and 2; 

76 

Inception phase (January 18 –March 24)Desk researchScoping interviewsFine-tuned methodologyPreliminary portfolio analysisScoping of the analysisData collection & analysis phase (March 24 –June 30)Desk researchInterviewsPreliminary findings on relevance and coherencePortfolio & composition analysisTargeted  consultationsCase studiesInterviewsSecondary data analysisFindings & conclusions at the Thematic Area levelAggregated analysis at the programmelevelConclusions & recommendations at the programmelevelWorkshopReporting phase (July 1 –October  6)Public consultationIndicator definitionInception reportInterim reportFinal report 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Type of document 

M&E available at the 
programme level 

Documents and 
sources for 
programmes prior to 
COSME – CIP: 

EU Policy documents 

EU Policy Studies 

Source: Technopolis 

Examples of relevant documents 
Impact Assessment for the proposal for a COSME regulation: 
Impact Assessment for the proposal for a COSME regulation COM (2011)834; 
Impact Assessment for the proposal for a COSME regulation SEC(2011) 1452; 
Impact Assessment for the proposal for a COSME regulation SEC(2011) 1453; 
COSME work programmes: 2014, 2015 and 2016; 
COSME monitoring reports 2014 & 2015 (2016 under development during the study) 

Impact assessment (2005); Interim evaluation (2010); Final evaluation (2011); 
CIP performance review 2007-2012; 
Implementation, evaluation, performance and beneficiaries reports for each year of 2007 
– 2012: http://ec.europa.eu/cip/documents/implementation-reports/index_en.htm 
Decision 1639/2006/EC of the European Parliament and of the Council of 24 October 2006 
establishing a Competitiveness and Innovation Framework Programme (2007-2013); 

EC (2014), For a European Industrial Renaissance, COM/2014/014 final  
EC (2012), A Stronger European Industry for Growth and Economic Recovery Industrial 
Policy Communication Update,  COM/2012/0582 final  
EC (2010), An Integrated Industrial Policy for the Globalisation Era Putting 
Competitiveness and Sustainability at Centre Stage, COM/2010/0614 final  
EC (2008, 2011), Small Business Act 
EC (2015), Single Market Strategy 
EC (2016), Europe's next leaders: the Start-up and Scale-up Initiative,  COM/2016/0733 
final 
EC (2012), Entrepreneurship 2020 Action Plan 
EC (2011) Industrial Policy: Reinforcing competitiveness 
Pellegrin, J. et al (2015), EU Industrial Policy: Assessment of Recent Developments and 
Recommendations for Future Policies, CSIL, CASE, University of Milan, Economisti 
Associati, a study for: European Parliament, Directorate General for Internal Policies, 
Policy Department A: Economic and Scientific Policy 
Dhéret, C., Morosi, M. (2014)  Towards a New Industrial Policy for Europe, EPC Issue 
Paper No.78  
Sylvest, J. et al. (2014) How can European industry contribute to growth and foster 
European competitiveness?, Danish Technological Institute, A study for: European 
Parliament, Directorate General for Internal Policies, Policy Department A: Economic and 
Scientific Policy 

A concrete output of our desk research was the overview of the achievements of the KPIs 
that  are  listed  in  Appendix  A.  This  analysis  was  performed  in  the  context  of  our 
assessment  of  the  efficiency  of  the  COSME  programme,  specifically  related  to  the 
monitoring activities. 

The table below gives an overview of the types of documents used for the assessment of 
COSME against the evaluation questions and topics. 

Table 15 Evidence table for the desk research and literature review 

Nr. 
question 

Evaluation topics 

SO1 

SO2 

SO3 

SO4 

a
t
a
d

y
r
a
d
n
o
c
e
s

/

i

s
e
d
u
t
S

s
t
n
e
m
u
c
o
d

e
m
m
a
r
g
o
r
P

a
t
a
d

y
r
a
d
n
o
c
e
s

/

i

s
e
d
u
t
S

s
t
n
e
m
u
c
o
d

e
m
m
a
r
g
o
r
P

i

s
e
d
u
t
s

r
e
h
t
O

i

s
e
d
u
t
s

r
e
h
t
O

a
t
a
d

y
r
a
d
n
o
c
e
s

/

i

s
e
d
u
t
S

s
t
n
e
m
u
c
o
d

e
m
m
a
r
g
o
r
P

a
t
a
d

y
r
a
d
n
o
c
e
s

/

i

s
e
d
u
t
S

s
t
n
e
m
u
c
o
d

e
m
m
a
r
g
o
r
P

i

s
e
d
u
t
s

r
e
h
t
O

i

s
e
d
u
t
s

r
e
h
t
O

 IMPLEMENTATION STATE OF PLAY 

Rationale of the programme and its actions 
Objective hierarchy 
Link with actions funded under CIP 
Budget allocation over objectives 
Budget allocation over types of actions 
Budget allocation over types of calls and 

X  X  X  X  X  X  X  X  X  X 
X 
X 
X  X 
X  X 
X 
X 
X 
X 
X 
X 

X 
X 
X  X  X  X 
X 
X 
X 
X 
X 
X 

77 

 
 
 
 
 
 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

SO1 

SO2 

SO3 

SO4 

RELEVANCE 
Q2.1) 

Q2.2) 

instruments 
Budget allocation over types of stakeholders 

Ongoing validity of the rationale 
Link developments/changes in budget allocations 
Alignment needs/actions/stakeholders 
Deficiencies in alignment needs/actions 
The decision-making process for the actions 
undertaken 
The societal relevance of the programme  
The policy relevance of the programme 

Q2.3) 
EFFECTIVENESS 
Q3.1) 

Outputs so far 
Results so far 
Alignment with expectations 
Appropriate stakeholders targeted 
Appropriate stakeholders reached 
Appropriate policy mix 
Appropriate budget allocation 
Progress towards policy objectives 

Q3.2) 

Q3.3) 
EFFICIENCY 
Q4.1) 

Q4.2) 

Q4.3) 

COHERENCE 
Q5.1) 

Q5.2) 

Q5.3) 

Administrative costs / stakeholders 
Affordability / burden 
Simplification measures 
Administrative costs / implementation structure 
Cost-effectiveness 
Comparison across actions 
Time-to-grant / / use of funding instruments / 
distribution of funding 
Implementation / delegation 
Monitoring system 

Internal coherence - SO 
Internal coherence - communication instruments 
Internal coherence - programme 
Internal coherence - coordination process 
External coherence - EU 
External coherence - EU process 
External coherence - MS 
External coherence - MS process 

EU ADDED VALUE 
Q6.1) 

Q6.2) 
Source: Technopolis 

Importance of action at EU level 
Additionality 
Ongoing need for action at EU level 

X  X 

X  X 
X  X 

X 

X 

X 

X  X 
X 

X 

X 

X  X 
X 
X  X 

X  X 

X 

X 

X  X 
X 

X  X  X  X 

X 
X  X 
X 
X 
X 

X 
X 

X 

X 

X  X 

X 

X 

X  X 
X 

X 

X 

X 

X 
X 

X 

X 
X 
X 
X 
X 
X 

X 

X 
X 
X 
X 

X 

X 

X 

X 
X 

x 
X 

X 

X 
X 

X 
X 
X 
X  X 

X 

X 
X 

X 

X  X 

X  X 

X 
X  X 

X 

X 
X 
X  X 

X  X 
X 

X 
X  X 

X 
X 

X 
X 
X  X  X  X 
X 
X 
X 
X 
X 
X  X 

X  X 
X 

X 

X  X 
X  X 
X 

X  X  X 
X 

X 

X 
X  X 

X 
X 
X 

X 

X 

X 

2.2.  Details at the Thematic Areas level 

2.2.1. 

SO1 – Access to finance 

Desk  research  and  secondary  data  analysis  were  conducted  based  on  60  documents 
identified during the inception phase and data collection phase.  

Overall  the  exercise  of  gathering  the  documentation  was  challenging  as  EU  officials  did 
not have at end a knowledge database  concerning the “Access to Finance” actions. This 
resulted in collecting documents throughout the overall study and up to September 2017. 
All available documentation was carefully processed and related to other findings.  

The  collection  of  secondary  data  is  primarily  important  for  the  portfolio  analysis  as  well 
for assessment of the effectiveness and the efficiency of the actions. Secondary data on 
the LGF and EFG were obtained from EIF notably Quarterly reports from 2014 to end of 
2016, Annual reports 2014 to 2016, and Growth & Employment report 2015. 

78 

 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

DG GROW shared a selection of other monitoring data of both instruments: 

  Extraction of DG GROW commitments to both EFG and LGF 

  Extraction of remuneration and fees paid to EIF for both instruments 

The list of documents analysed is presented in Appendix A.1   

2.2.2. 

SO2 – Access to market 

One  of  first  steps  in  the  evaluation  was  the  collection  of  the  relevant  documentation 
for  the  desk  research  on  the  actions  under  SO2.  We  contacted  the  relevant  EC  officers 
with a documentation request related to the following actions: 

  Enterprise Europe Network  

  Your Europe Business portal  

 

Improved Points of Single Contact  

  Facilitating Access to Light Remotely Piloted Aircraft Systems (RPAS) Regulation  

  Exchanges of Good Practices in the Area of Compliance Assistance and Schemes  

 

Improving SMEs’ Access to Public Procurement  

  SME Internationalisation Portal  

  EU-Japan Centre  

  China, Latin America and South-East Asia IPR SME Helpdesks  

  Support to SME Internationalisation  

 

Industrial Policy Cooperation 

Most  secondary data were identified in a meeting with Unit H2 on 9 February 2017 and 
have reached us via DG GROW. Additional secondary data were sent by EASME.  

Overall  the  exercise  of  gathering  the  documentation  was  challenging,  as  many  EU 
officials  were  very  busy,  especially  within  the  holiday  period  that  overlapped  with  the 
short  time  period  between  the  inception  report  and  the  interim  report.  Furthermore, 
although many actions come from the Work Packages of 2014 and 2015, not all actions 
have  been  implemented  to  the  extent  that  was  initially  expected.  By  the  end  of  the 
evaluation, while information was made available on all the actions, for some actions the 
information  remained  limited.  All  available  documentation  was  carefully  processed  and 
related to other findings.  

The  collection  of  secondary  data  is  primarily  important  for  the  portfolio  analysis  as 
well for assessment of the effectiveness and the efficiency of the actions. Secondary data 
on the Enterprise Europe Network were obtained from EASME. for;  

The  largest  source  of  collected  information  is  the  Performance  Enhancement  System 
(PES)  database,  which  relies  on  regular  reporting  done  by  the  Network  members.  The 
aim of the PES is to measure the achievements of the Network members, and to monitor 
the  overall  Network  performance.  It  includes  an  Inception  Report  (6  months  after  the 
start  of  the  Consortium),  quantitative  reporting  on  the  16  key  variables  of  the  PES  (12 

79 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

months, as well as 24 months after the start of the Consortium), self-assessments by the 
Consortium, as well as some other reflective elements.86 

It  often  takes  quite  some  time  to  finalise  all  the  data  for  the  PES,  therefore  the  time 
available  to  prepare  the  data  within  the  evaluation  was  short.  A  selection  of  the 
preliminary  PES  data  that  were  available  in  time  to  be  included  in  the  final  report  was 
provided to the evaluation team. EASME also shared a selection of other monitoring data 
of the Network, namely: 

  Extraction of Achievements, providing insight into the number of Partnership 

Agreements and Advisory Service Outcomes realised per consortium 

  Extraction of the Partnership Opportunity Database, providing insight into the number 

of partnering opportunities that were published per consortium, as well as the 
number of expressions of interest that those profiles generated 

  Extraction of the new Impact Survey results. This is a new survey and therefore the 
results are still incomplete. It does provide insight into the way impact data are 
currently collected and what kinds of data are produced. 

The list of documents analysed is presented in Appendix C.2   

2.2.3. 

 SO3 – Favourable environment 

Desk research has been completed for all actions, and has aimed to provide an overview 
of the economic developments and policy issues within the different thematic areas under 
SO3: SME Policy, regulatory and administrative burden, innovation performance in SMEs, 
digitisation and KETs, as well as sectoral competitiveness.   

The  secondary  data  we  used  came  from  the  COSME  Work  Programmes  2014-2015  and 
the COSME Activity Reports 2014 and 2015 for all actions in focus. Where available, Calls 
for  Proposals  /  Projects  for  the  various  streams  of  actions  (especially  for  the  cluster 
internationalisation actions, tourism, and e-skills actions), and technical progress reports 
or  final  implementation  reports  were  consulted.  Moreover,  for  SME  Policy,  the  SME 
Performance  Review  publications,  documents  published  by  SME  Envoys  and  the 
description of the mission of the SME Envoys were used. 

Beyond the COSME work programme and activity reports, the level of access, utility and 
comprehensiveness of further secondary data varies per field. In some cases, it has been 
difficult  to  obtain  a  comprehensive  overview  of  the  status  of  the  implementation  and 
interim  results  achieved,  especially  for  the  measures  where  the  effectiveness  dimension 
was  in  focus.  In  particular,  synthesis  reports  of  the  projects  funded  under  the  Cluster 
Internationalisation call were  only partially available, as the data collection for the first-
year  performance  review  had  not  been  completed  by  the  time  of  the  evaluation,  as 
several cluster partnerships had yet to submit their second progress reports. In the case 
of  the  tourism  actions,  no  synthesis  reports  of  the  individual  project  monitoring  and 
activity  reports  were  available  and,  due  to  their  non-standardised  format,  individual 
progress  reports  could  not  be  used  to  provide  a  more  comprehensive  overview  of  the 
progress of actions and outputs in the Tourism sector. To counterbalance the lack of data 
in these cases, the results of the survey with beneficiaries of these two actions were used 
to illustrate the level of progress achieved in a more aggregated manner.  

86 See also: EASME (2014). Guide for Applicants to COS-WP2014-2-1 

80 

 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

The list of documents analysed is presented in Appendix C.3   

2.2.4. 

SO4 – Entrepreneurship 

Desk  research  activities  have  been  ongoing  throughout  the  evaluation,  but  were  a 
particular focus of the first stages of the study, to inform consultation activities and early 
analysis. The evaluation team collected and analysed secondary sources in relation to all 
the  entrepreneurship  actions  in  scope.  Information  was  sought  from  the  European 
Commission  and  EASME  to  ensure  that  both  technical  and  policy-relevant  information 
was  considered.  In  addition,  online  searches  and  interviews  were  used  to  complete  the 
review. 

Additional information was obtained in relation to the key entrepreneurship action (EYE). 
This included information from the  EYE programme database (containing information on 
all  intermediaries  and  participants  involved  in  EYE  placements),  past  evaluations  (the 
interim evaluation of the EYE pilot and the EYE statistical analysis), a selection of recent 
six-monthly  monitoring  reports  produced  by  the  EYE  support  office  (Eurochambres)  and 
the EYE programme guide and quality manual. 

Information  on  the  individual  actions  was  captured  in  a  structured  format  to  facilitate 
analysis  along  the  main  evaluation  questions.  These  secondary  sources  are  drawn  upon 
and  referenced  throughout  the  report,  but  particularly  when  analysing  the  context, 
relevance and coherence of the entrepreneurship actions. 

The list of documents analysed is presented in Appendix C.4   

81 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

3.  STATISTICAL DATA ANALYSIS  

In  this  chapter  we  begin  with  an  overview  of  our  sources  of  information  (Section  3.1). 
Then  we  present  the  portfolio  analysis  (Section  3.2),  the  composition  analysis  (Section 
3.3), the programme management data (Section 3.4) and the cost-effectiveness analysis 
(Section 3.5). 

3.1.  Sources 

Ten structured databases have been identified for the programme and the different SOs, 
based on input provided by the officials in charge of the databases at DG GROW, EASME 
and EIF (Table 16, below). Contacts with these officers and the EC officers acting as the 
SO  contact  persons  continued  throughout  the  study  for  the  retrieval  of  some  more 
detailed information and the completion of this list. 

These  datasets  provided  input  for  the  composition  and  portfolio  analysis  depicting  the 
programme’s ‘state-of-play’ as well as for the other analyses, especially those related to 
the  relevance,  effectiveness  and  efficiency  questions.  Some  may  serve  also  for  the 
targeted consultations. 

Table 16 Preliminary list of the available relevant databases 

Database 
1 
2 

Name 
Data hub for COSME 
Data from Annual 
operational reports 

SO 
SO2, 3, 4 
SO1 

Data from the Growth 
and employment report 
2015 (LGF) 
Data from Quarterly 
reports 

Data from EIF 

SO1 

SO1 

SO1 

Achievement database 

SO2 

3 

4 

5 

6 

7 

Performance 
Enhancement System 
Various data for EEN 
EYE database 
Statistics 

SO2 

SO2 
SO4 
SO1, 2, 3, 4 

8 
9 
10 
Source: Technopolis 

Level 
Projects 
Aggregated + 
financial 
intermediaries 
Aggregated 

Aggregated + 
financial 
intermediaries 
Intermediaries and 
recipients 
Partnership 
agreements and 
Advisory service 
outcomes 
Consortia and 
network partners 
/ 
HE, NE, EP, IO 
Countries 

Contact 
EASME 
DG GROW 

DG GROW 

DG GROW 

EIF 

EASME 

EASME 

EASME 
DG GROW 
Publicly available 

3.2.  Portfolio analysis 

For  the  portfolio  analysis,  both  the  data  on  planned  and  committed  budgets  were 
instruments, 
collected.  The  budgetary  distribution  was  calculated  across  policy 
implementation  modes,  and  thematic  areas  (and  corresponding  action  lines)  and 
combinations thereof.  

The outcomes of this analysis especially informed: 

 

the analysis of the relevance of the programme, in terms of the budget allocated for 
the  attainment  of  specific  objectives  and  sub-objectives  as  an  illustration  of 
importance attributed  

82 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

 

 

the analysis of  effectiveness, in terms  of the  level  of the  specific effects that can be 
expected 

the analysis of implementation efficiency, in terms of the extent to which the COSME 
programme  allocated  sufficient  levels  of  budget  to  specific  actions  and  the  projects 
within  these  actions,  and  the  eventual  fragmentation  of  the  budget  over  multiple 
actions with similar objectives 

Table 17 Portfolio analysis levels 

Policy instruments 
Financial instrument  
 Information instruments  
 Knowledge & networking 
instruments 

Implementation modes 
Ad hoc grant Art. 190 RAP 
Administrative arrangement 
Calls for tender (service contracts) 
Direct contracts 
FWC (specific contracts) 
Grants 
Indirect Management 
Membership fees 
Not defined 
Reimbursement experts / members 
Remuneration experts / evaluators 
Specific grant agreement under 
Framework 
Partnership Agreement 

Thematic areas 
Business mgt capacity  
Digitisation & KETs 
Entrepreneurship 
Information campaign 
Internationalisation 
Regulations & administrative burden 
Responsible innovation 
Sectoral competitiveness 
SME policy 
Support to access finance 

Source:  for  financial  instruments:  EIF  quarterly  report  Q4/2016;  for  EEN  and  EYE:  databases  provided  by 
COSME 

3.3.  Composition analysis 

The  composition  analysis,  i.e.  the  profiling  of  the  stakeholders  involved  in  COSME,  has 
been  conducted;  however,  the  predominant  use  of  agreements  for  the  allocation  of  the 
COSME  budget,  in  particular  their  delegation  agreements  (indirect  management)  or 
Framework  Partnership  Agreements,  constituted  a  challenge  for  the  identification  and 
analysis of the final beneficiaries’ profile. 

This  analysis  was  therefore  conditioned  by  the  integration  of  the  COSME  data  hub  with 
the  databases  related  to  the  specific  actions  (financial  instruments,  EEN,  EYE,  etc.)  as 
well as, most importantly, by the availability of detailed information on the stakeholders 
involved,  covering  the  intended  dimensions  for  analysis.  Particularly  in  the  case  of  the 
type  of  stakeholders  involved  the  original  databases  were  insufficient  for  composition 
analysis  due  to  the  lack  of  a  predetermined  typology  of  stakeholders.  To  amend  this, 
database  entries  were  manually  checked  in  order  to  attribute  stakeholders  to  the 
typology  designed  for  this  evaluation  (i.e.  industry,  industry  association,  association, 
public administration/agency, research/RTO). 

Similarly,  the  sectoral  representation  in  the  original  databases  was  insufficient  for  the 
composition  analysis  due  to  the  absence  of  a  systematic  sectoral  identifier,  such  as  the 
NACE  rev.2  classification,  which  was  only  available  for  SO1.  To  amend  this,  a 
concordance  with  NACE  rev.2  codes  was  developed.  It  should  be  noted,  however,  that 
the concordance provides only for a rough approximation, given that some sectors do not 
correspond  to  a  NACE  rev.2  classification  (e.g.  Organic/Bio/Eco  products  and  services, 
etc.) or some of the entries were too broad and required a matching with multiple NACE 
rev.2  levels  (e.g.  Mining,  machinery  for  mining,  basic  metals  and  related  products:  B  + 
C28.92 + C24). 

This  analysis  was  therefore  conditioned  by  the  integration  of  the  COSME  data  hub  with 
the  databases  related  to  the  specific  actions  (financial  instruments,  EEN,  EYE,  etc.)  as 
well as, most importantly, by the availability of detailed information on the stakeholders 

83 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

involved,  covering  the  intended  dimensions  for  analysis.  Dimensions  for  analysis  were 
the  geographical  locations  of  the  beneficiaries,  as  well  as  the  types  of  stakeholders 
involved in the actions funded and their sectoral representation. 

This analysis encountered difficulties in terms  of data availability related to the types  of 
stakeholders and sectors of activity. No data was available in relation to gender, except 
for the EYE participants. 

The information available by Action Line is described in Table 18 and Table 19, below. 

Table 18 Overview of available data on beneficiaries  

SO 

Location 

Financial instruments 

Yes 

Type of 
stakeholder 
Yes 

EEN services 

Yes 

No 

Sector 

Yes 

Yes 

Cluster 
Internationalisation/ 
Tourism (grants) 

EYE 

Yes for 
type of 
call=calls 
for 
proposals 
Yes 

Yes 

Yes 

Yes for type of call=calls 
for proposals 

Yes for Host 
entrepreneurs. It is not 
applicable for New 
entrepreneurs.  

Gender 

No 

No 

No 

Yes 

Budget 

Yes 

Not applicable: EEN 
network member 
organisations receive 
funding to provide 
services to final 
beneficiaries 
Yes for type of 
call=calls for proposals 

Yes for New 
Entrepreneurs. This is 
a fixed monthly 
amount adjusted by 
country. Host 
Entrepreneurs do not 
receive funding. 

Source:  for  financial  instruments:  EIF  quarterly  report  Q4/2016;  for  Cluster  Internationalisation:  European 
Cluster Collaboration Platform; for EEN and EYE: databases provided by COSME; for Tourism: COSME data-hub.  

Table 19 Overview of available data on intermediaries  

Location (country) 
Yes 

Type of stakeholder 
Yes 

SO 
Financial 
instruments 
EEN 
services 
EYE 
Source:  for  financial  instruments:  EIF  quarterly  report  Q4/2016;  for  EEN  and  EYE:  databases  provided  by 
COSME. 

Gender 
No 

Budget 
Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

Additional analyses on geographical distribution and sectoral involvement are available in 
Appendix B. 

3.4.  Analysis of COSME programme management and proposal data 

The  time  to  grant  is  measured  as  the  number  of  months  between  the  deadline  for  the 
submission  of  proposals  and  the  signature  of grants for  successful  proposals.  The 
empirical scope of the time-to-grant analyses covers the calls listed in Table 20. 

Table 20 Calls on which time to grant analysis is based  

Business mgt capacity  
EEN Administration & services 

COS-Adhoc-2014-2020 
COS-EEN-2014-2-01 FPA 
COS-EEN-2014-2-04 

84 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

COS-EEN-SGA-16-B-07-2016-1 SCALE UP CALL 
COS-EEN-SGA2-2016-2-01 
EEN-SGA-COSME-2015-01-1 

Entrepreneurship 

Entrepreneurship education 

COS-ENTEDU-2014-4-06 

EYE 

COS-EYE-2014-4-05 
COS-EYE-2015-4-01 
COS-EYE-FPA-2016-4-01 
COS-EYE-SGA-2016-4-01 

Migrants 

COS-MigrantsENT-2016-4-02 

Second starters 

COS-EarlyWarningEU-2016-4-01 

Internationalisation 

Cluster internationalisation 
COS-CLUSTER-2014-3-03 

IPR Helpdesks 

COS-IPR-2014-2-05 

SME Internationalisation 
COS-ETPO-2015-2-01 

Regulations & administrative burden 

Access to public procurement 

COS-APP-2016-2-05 
Access to regulations 
COS-RPAS-2014-2-03 

Privacy 

COS-DRONES-2016-03-02 

Single points of contact 
COS-SPOC-2016-02-03 
Sectoral competitiveness 

Clusters 

COS-CLUSTER-2014-3-04-02-1 
COS-CLUSTER-2015-3-02 

Design-based consumer goods 

COS-DESIGN-2015-3-03-1 
COS-DESIGN-2015-3-06 

Retail 

COS-IPPDS-2015-2-02-1 

Tourism 

COS-TEDEN-2014-3-15 
COS-TEDEN-2015-3-05 
COS-TFLOWS-2014-3-15 
COS-TOUR-2015-3-04-1 
COS-TSUST-2014-3-15 
COS-TSYNER-2014-3-15 

SME policy 

SBA implementation 

COS-DCFB-2015-3-01-1 

Source: Technopolis elaboration based on raw data provided by COSME 

The table below presents a complete and detailed time-to-grant analysis for each action 
line (compared to the tables in the core body  of the report,  where only the main action 
lines  are  listed).  For  transparency  purposes,  the  table  includes  also  the  action  lines  for 
which  data  are  not  available,  in  order  to  provide  a  complete  overview  of  the  data 
provided. 

85 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Table 21 Time to grant by thematic area and action line 

Themes 

Action lines 

Average 
Time to 
grant in 
days 

Average Time to 
grant in months 
(assumption 
month=30 days) 

Min Time to 
grant in 
months 
(assumptio
n month=30 
days) 

EEN Administration 
& services 

Business mgt 
capacity  
Business mgt capacity  overall average 
Entrepreneurship 
Entrepreneurship 
education 
EYE 
Migrants 
Second starters 

Entrepreneurship overall average 
Internationalisation 

Cluster 
internationalisation 
IPR Helpdesks 
SME 
Internationalisation 

Internationalisation overall average 
Regulations & 
administrative 
burden 

Access to public 
procurement 

Access to 
regulations 
Privacy 
Single points of 
contact 

Regulations & administrative burden 
overall average 
Sectoral 
competitiveness 

Clusters 

Design-based 
consumer goods 
Retail 
Tourism 

234.8 

234.8 
209 

218.6 

178 
217.6 
222.6 

190.7 
114.5 

205.3 
277.5 

226 

147 

247.2 

181.5 

235.4 

174 
250 

Sectoral competitiveness overall average  239.9 
215.4 
SBA 
SME policy 
implementation 

7.8 

7.8 
7.0 

7.3 

5.9 
7.3 
7.4 

6.4 
3.8 

6.8 
9.3 

7.5 

4.9 

8.2 

6.0 

7.8 

5.8 
8.3 
8.0 
7.2 

SME policy overall average 
Grand Total (overall average for all 
thematic areas) 
Source: Technopolis elaboration based on raw data provided by COSME 

215.4 
232.8 

7.2 
7.8 

3.9 

3.9 
7.0 

3.8 

5.9 
3.8 
6.7 

2.7 
3.7 

2.7 
8.3 

7.5 

4.9 

4.9 

5.4 

7.6 

5.8 
5.6 
5.4 
6.1 

6.1 
2.7 

Max Time 
to grant in 
months 
(assumptio
n 
month=30 
days) 
26.8 

26.8 
7.0 

10.0 

5.9 
10.0 
8.0 

10.0 
4.0 

10.0 
10.5 

7.5 

4.9 

10.5 

7.0 

8.3 

5.8 
14.1 
14.1 
9.1 

9.1 
26.8 

3.5.  Cost-Effectiveness analysis 

Our  indicators  reflect  the  number  of  outputs  and  outcomes  produced  by  the  (million) 
Euros invested (e.g. “how many SMEs have been effectively supported by Euro spent?”). 
This approach provides metrics on efficiency that can be compared across actions. 

To  conduct  the  analysis,  we  have  brought  together  the  costs  and  benefits  reported  for 
each  action.  The  analysis  focus  on  four  main  actions:  LGF  (Access  to  finance),  EEN 
(Business  Management  Capacity),  Cluster  Go  International  (Internationalisation),  and 
EYE  (Entrepreneurship).  Together  these  four  actions  account  for  62%  of  the  budget 
committed to COSME in 2014-2016 (EUR 547 million out of EUR 879 million). 

In terms of costs, we have considered the (i) direct cost to the programme (i.e. budget 
committed  to  COSME  in  2014-2016  and  administrative  costs  paid  to  EASME  and  EIF), 
and (ii) the indirect cost to beneficiaries to account for the  cost to the ‘system’.   In the 
case  of  EEN  and  Cluster  Go  International  this  includes  the  co-funding  from  partner  / 
member  organisations;  in  the  case  of  EYE  this  includes  additional  costs  incurred  by  the 

86 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

host  and  new  entrepreneurs;  and  in  the  case  of  LGF  this  includes  the  additional 
guarantee  provided  under  EFSI.  Two  measures  of  CEA  indicators  have  been  calculated 
based  on  those  costs  (with  and  without  indirect  costs).  Only  the  indicators  that  include 
indirect costs are presented in the Main Report.  

In  terms  of  benefits,  we  have  arrived  at  grossed-up  estimates  of  benefits  considering 
information  from  existing  monitoring  data  (such  as  the  population  of  beneficiaries)  and 
the results obtained from our surveys.  

As  such,  the  analysis  does  not  pretend  to  be  exhaustive.  In  terms  of  cost,  the  analysis 
focuses  mainly  on  the  costs  and  investments  sustained  by  the  EC  and  participant 
companies  /  organisations.  The  analysis  does  not  consider  the  potential  costs  to  wider 
society.  Similarly,  the  analysis  of  benefits  focuses  on  the  impact  on  participant 
organisations and does not include the analysis of externalities (i.e. the potential benefits 
on  organisations  and  on  communities  not  directly  involved  in  any  of  the  specific 
programmes). 

We have mainly used four sources of information in our analysis: 

 

 

Information on budget committed (and additional EFSI guarantee used in the case of 
LGF) 

Information on administrative costs paid to EASME (proportionally distributed across 
actions to equate to 6.6% of budget committed) and EIF  

  Monitoring data from the different actions, which have provided figures on total 

participations and have been used to arrive at grossed-up estimates. 

  Our beneficiary surveys (including, the LGF SME survey, EEN SME client survey, 

Cluster organisation survey, HE surveys and NE surveys), which have provided key 
parameters for our estimations, including indirect costs 

We  have  reached  estimates  for  a  total  of  22  indicators.  The  section  below  provides  full 
overview of the costs and benefits mapped for each of those actions. In the case of LGF 
we have estimated 8 additional indicators to account for the fact that the current 2014-
2016 budget envelope (from COSME and EFSI guarantee) will continue to deliver results 
as  the  financial  intermediaries  have  2  to  3  years  to  build  up  loan  portfolios  under  the 
guarantee  agreements  signed  under  2014-2016  budget  envelope.  So  far,  a  total  of 
143,000 SMEs has been supported, and it is expected that a total of  300,000 SMEs will 
be supported under guarantee agreements signed with financial intermediaries within the 
same budget envelope. 

87 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Table 22  Cost-effectiveness analysis 

Note that any discrepancies between the parameters and the final figures are due to rounding. 

CEA Indicator  

Costs 

Benefits 

Description 

Direct (Budget 
committed)  
Value (mEUR ) 
[+ 
administrative 
fees] 

Value / unit 

Indirect  
Value (mEUR 
) (Cost to 
the system) 

[A] 

[B] 

[C] 

Support to access finance – LGF (achieved so far) 

COSME Budget 
committed: 
349.2 mEUR   
[+26.3 mEUR ] 

EFSI 
guarantee: 
262.4 mEUR   

143,344 SMEs 

SMEs supported so far as reported in monitoring data 
90,307 SMEs 

[1] Number of SMEs supported through 
COSME financing, per EUR  million 
invested 
[17] Number of SMEs supported 
through COSME financing (that would 
not have had access to other sources 
of funding), per EUR  million invested 

[18] Value of credits leverage, per EUR  
million invested 

[19] Value of private funds leverage, 
per EUR  million invested 

[5] Increased turnover by companies 
that had access to LGF (in EUR ), per 
million EUR  invested 

No data on 
the value of 
costs for 
SMEs has 
been 
collected 
however 
evidence 
suggests 
that 
additional 
costs to 
SMEs are 
small 

143,344 SMEs multiplied by the percentage of SMEs that stated that the 
financing supported by a EU-COSME guarantee was the only option available 
or that other options were available but did not cover the full amount (63% 
according to the LGF SME survey) 
5,547.2 mEUR  

Value of Principal Amount as reported in monitoring data 
1,247.6 mEUR  

Grossed-up figures based value of private funds leveraged, as a percentage 
of Principal Amount (as reported by survey respondents) multiplied by the 
value of the Principal Amount, considering size of the company. 
10,653 mEUR  

28.4 

16.7 

Average value of turnover at the time of first inclusion (362 kEUR ), 
multiplied by the average growth on turnover (20. 5%) as reported in the 
“COSME Loan Guarantee Facility – Employment and Growth report, 2015”. 
This figure is then grossed-up by multiplying the average value of the 
increased employment with the total number of SMEs (143,344) 

88 

CEA  
Indicator 
I 
Value /unit 
per EUR  
million 
invested 

[C]/ [A] 

CEA 
Indica
tor II 
Value 
/unit 
per 
EUR  
million 
investe
d 
[C] / 
([A] + 
[B]) 

381.7 

224.7 

240.5 

141.6 

14.8 

8.7 

3.3 

2.0 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

CEA Indicator  

Costs 

Benefits 

[11] Increased employment by 
companies that had access to LGF, per 
million EUR  invested 

COSME Budget 
committed: 
349.2 mEUR   
[+26.3 mEUR ] 

[8] (Additional) Increased turnover by 
companies that had access to LGF (in 
EUR ), per million EUR  invested 

[14] (Additional) Increased 
employment by companies that had 
access to LGF, per million EUR  
invested 

EFSI 
guarantee: 
262.4 mEUR   

No data on 
the value of 
costs for 
SMEs has 
been 
collected 
however 
evidence 
suggests 
that 
additional 
costs to 
SMEs are 
small 

Support to access finance – LGF (expected) 

[1b] Number of SMEs supported 
through COSME financing, per EUR  
million invested 

COSME Budget 
committed: 
349.2 mEUR   
[+26.3 mEUR ] 

EFSI 
guarantee: 
262.4 mEUR   

[17b] Number of SMEs supported 
through COSME financing (that would 
not have had access to other sources 
of funding), per EUR  million invested 

[18b] Value of credits leverage, per 
EUR  million invested 

[19b] Value of private funds leverage, 

No data on 
the value of 
costs for 
SMEs has 
been 
collected 
however 
evidence 
suggests 
that 
additional 
costs to 
SMEs are 
small 

CEA  
Indicator 
I 
632.3 

CEA 
Indica
tor II 
372.2 

237,431 people 

Average number of employees at the time of first inclusion (3.9), multiplied 
by the average growth on employment (42.7%) as reported in the “COSME 
Loan Guarantee Facility – Employment and Growth report, 2015”. This figure 
is then grossed-up by multiplying the average value of the increased 
employment with the total number of SMEs (143,344) 

6,711.3 mEUR  

17.9 

10.5 

Grossed-up value of increased turnover (as shown in indicator [5]) 
multiplied by % of SMEs that stated that ‘the financing supported by a EU-
COSME guarantee was the only option available or that other options were 
available but did not cover the full amount’ (63% according to the LGF SME 
survey) 
149,581 people 

Grossed-up value of increased employment (as shown in indicator [11]) 
multiplied by % of SMEs that stated that the financing supported by a EU-
COSME guarantee was the only option available or that other options were 
available but did not cover the full amount (63% according to the LGF SME 
survey) 

398.3 

234.5 

300,000 SMEs expected to be supported 

798.9 

470.3 

As estimated by the European Commission. All the estimates below use a 
rule of three to recalculate the estimates based on 143,000 SMEs. In 
practice, all the above indicators are multiplied by a factor of 2.09. 
189,000 SMEs 

503.3 

296.3 

300,000 SMEs multiplied by % of SMEs that stated that the financing 
supported by a EU-COSME guarantee was the only option available or that 
other options were available but did not cover the full amount (63% 
according to the LGF SME survey) 
18,904.7 mEUR  

Based on EIF Q4 2016 operational report. A simple ‘rule of three’ gives a 
value of 11bnEUR , but we use the 18bn figure as it considers the real 
implementation conditions under each Guarantee Agreement with the 
respective Financial Intermediaries and (ii) it is based also on a previous 
experience gained in the implementation of similar debt financial 
instruments.  
2,611.0 mEUR  

50.3 

29.6 

7.0 

4.1 

89 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

CEA Indicator  

Costs 

Benefits 

CEA  
Indicator 
I 

CEA 
Indica
tor II 

per EUR  million invested 

[5b] Increased turnover by companies 
that had access to LGF (in EUR ), per 
million EUR  invested 

[11b] Increased employment by 
companies that had access to LGF, per 
million EUR  invested 

[8b] (Additional) Increased turnover by 
companies that had access to LGF (in 
EUR ), per million EUR  invested 

[14b] (Additional) Increased 
employment by companies that had 
access to LGF, per million EUR  
invested 

Grossed-up figures based on the average value of private funds leveraged by 
survey respondents and number of SMEs expected to be supported 
(300,000), considering size of the company 
22,295.1 mEUR  

59.4 

34.9 

Average value of turnover at the time of first inclusion (362 kEUR ), 
multiplied by the average growth on turnover (20. 5%) as reported in the 
“COSME Loan Guarantee Facility – Employment and Growth report, 2015”. 
This figure is then grossed-up by multiplying the average value of the 
increased employment with the total number of SMEs expected to be 
supported (300,000) 
496,911 people 

Average number of employees at the time of first inclusion (3.9), multiplied 
by the average growth on employment (42.7%) as reported in the “COSME 
Loan Guarantee Facility – Employment and Growth report, 2015”. This figure 
is then grossed-up by multiplying the average value of the increased 
employment with the total number of SMEs expected to be supported 
(300,000) 
14,045.9 mEUR  

Grossed-up value of increased turnover (as shown in indicator [5]) 
multiplied by % of SMEs that stated that ‘the financing supported by a EU-
COSME guarantee was the only option available or that other options were 
available but did not cover the full amount’ (63% according to the LGF SME 
survey) 
313,054 people 

Grossed-up value of increased employment (as shown in indicator [11]) 
multiplied by % of SMEs that stated that the financing supported by a EU-
COSME guarantee was the only option available or that other options were 
available but did not cover the full amount (63% according to the LGF SME 
survey) 

1,323.2 

778.9 

37.4 

22.0 

833.6 

490.7 

Business management capacity - EEN 

[2] Number of SMEs reached and 
supported, per EUR  million invested 

[6] Increase in turnover linked to 
services provided by EEN (in EUR ), per 
million EUR  invested 

COSME Budget 
committed: 
144.5 mEUR   
[+ 9.6 mEUR ] 

57.8 mEUR  
[Based on 
co-funding 
from EEN 
members, 
equivalent to 
40% of the 
EC 
contribution] 

210,000 SMEs 

Estimations prepared in the context of this evaluation 
34,785 mEUR  

1363 

991 

949 

690 

Grossed-up figure of turnover based on average turnover growth per SMEs 
in the period 2014-2016, as reported in the EEN SME client survey (696 
kEUR ), multiplied by the total number of SMEs supported (50,000 SMEs). 

90 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

CEA Indicator  

Costs 

Benefits 

[12] (Additional) Increase in turnover 
linked to services provided by EEN (in 
EUR ), per million EUR  invested 

[9] Increase in employment linked to 
services provided by EEN, per million 
EUR  invested 

[15] (Additional) Increase in 
employment linked to services 
provided by EEN, per million EUR  
invested 

1,739.2 mEUR  

Grossed-up figure of turnover (as shown in indicator [6]) multiplied by an 
additionality factor of 6.5%.  
The additionality factor is based on our calculations on growth presented in 
EEN Thematic Report, where we show that turnover among EEN SME clients 
has grown 6.5 percentage points more than a benchmark group. This 
estimate excludes outliers and considers death rates. 
200,000 employees 

Grossed-up figure of employment based on average growth on number of 
employees per SMEs in the period 2014-2016, as reported in the EEN SME 
client survey (4 multiplied by the total number of SMEs supported (50,000 
SMEs). 
19,000 employees 

Grossed-up figure of turnover (as described above in indicator [9]) 
multiplied by an additionality factor of 9.5%. 
The additionality factor is based on our calculations on growth presented in 
EEN Thematic Report, where we show that employment among EEN SME 
clients has grown 9.5 percentage points more than a benchmark group. This 
estimate excludes outliers and considers death rates. 

CEA  
Indicator 
I 
2727 

CEA 
Indica
tor II 
1983 

62 

45 

518 

377 

Internationalisation - Cluster Go International 

[20] Number of cluster supported, per 
EUR  million invested 

2.3 mEUR  
[+ 0.1 mEUR ] 

[3] Number of SMEs supported by 
Cluster, per EUR  million invested 

Entrepreneurship - EYE 

[4] Number of SMEs supported, per 
million EUR  invested 

24.3 mEUR   
[+ 1.6 mEUR ] 

[21] Number of new entrepreneurs 
that have created their own business 
(thanks in part to their participation in 
EYE), per million EUR  invested 

0.6 mEUR   
[Based on 
co-funding 
from 
partners, as 
funded ESCP 
received 
75% co-
funding from 
the EC] 

12.2 mEUR   

[Based 
Calculated 
based on the 
information 
provided by 
HE and NE 

93 

Based on number of partners as reported in ESCP-4i website 
11,246  

Based on estimated number of SMEs supported in Strand 1 and 2, according 
to ESCP-4i 2nd synthesis report 

38.8 

31.4 

4,998.2 

3,798.
0 

3,737 

144.4 

98.0 

2,117 New entrepreneurs and 1,620 Host Entrepreneurs, as reported in 
monitoring data 
241 New entrepreneurs 

9.3 

6.3 

Based on percentage of NEs creating a business according to our survey 
(11.4%), multiplied by the number of new entrepreneurs (2,117). Our 
calculation (11.4%) includes those NE that have reported turnover in 2016 

91 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

CEA Indicator  

Costs 

Benefits 

(ie had a business generating turnover in 2016). 

CEA  
Indicator 
I 

CEA 
Indica
tor II 

via the 
survey, and 
the number 
of HE and NE 
involved in 
the EYE in 
the period of 
analysis] 

[22] Number of entrepreneurs 
engaging in at least one business 
cooperation agreement, per million 
EUR  invested 

[7] Increase in turnover linked to EYE 
(in EUR ), per million EUR  invested 

[13] Increase in employment linked to 
EYE, per million EUR  invested 

[10] (Additional) Increase in turnover 
linked to EYE (in EUR ), per million EUR  
invested 
[16] (Additional) Increase in 
employment linked to EYE, per million 
EUR  invested 

Source: Technopolis (2017) 

993 Entrepreneurs 

38.4 

26.0 

Based on percentage of NE engaging in at least one business cooperation 
agreement according to our survey (46.9%) multiplied by the number of 
new entrepreneurs (2,117) 
209.0 mEUR  

8.1 

5.5 

Grossed-up turnover figures reported in the HE and NE surveys, and number 
of HE and NE.  The figure includes the increase in total turnover for both HE 
(203.8 mEUR ) and NE (5.2 mEUR ). The estimated average increase in 
turnover for HE is 125.7 kEUR  and for NE is 2.5 kEUR ] 
122.2 mEUR  

85.6 

58.1 

Based on grossed-up figure of turnover and an additionality factor of 58% 
and 73% for HE and NE respectively. These factors are calculated based on 
survey responses. 58%of HE that agree that NE contributed to growth in 
companies' turnover, while 73% % of NE that agree that EYE has helped 
them to establish a new business. We use these parameters and assume 
that in 58 per cent of the cases attribution for growth is 100% while in 47 
per cent attribution is 0%. A similar assumption is applied to NE 
2,216 employees 

Grossed-up turnover figures reported in the HE and NE surveys 
1,247 employees 

Based on grossed-up figure of turnover and an additionality factor of 41% 
and 73% for HE and NE respectively. These factors are calculated based on 
survey responses. 41%of HE that agree that NE contributed to growth in 
companies' employment. In the case of NE we use the same measure for 
turnover. We use these parameters and assume that in 41 per cent of the 
cases attribution for growth is 100% while in 59 per cent attribution is 0%. A 
similar assumption is applied to NE. 

4.7 

3.2 

48.2 

32.7 

92 

 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

4.  THE CONSULTATIONS  

This  chapter  presents  the  different  types  of  consultation  we  used.  We  begin  with  the 
quantitative  targeted  consultations  (4.1):  first  an  overview,  then  the  sampling  strategy, 
then  the  results  for  each  SO.  In  section  4.2  we  present  the  qualitative  targeted 
consultations.  The  last  section  deals  with  the  public  question  (0):  its  structure,  its 
implementation, the outcomes and its limits. 

4.1.  The targeted consultations – quantitative 

4.1.1. 

Overview  

This  section  presents  an  overview  of  the  targeted  consultation,  specifically  the  seven 
‘quantitative’ surveys that have informed this evaluation.  

Table  23  presents  an  overview  of  the  surveys,  including  the  target  group,  population, 
target  sample,  responses  and  response  rates,  while  a  more  detailed  reporting  on  the 
sampling  methods,  data  collection  process  and  response  rates  is  provided  in  the  sub-
sections below. 

All  draft  questionnaires  were  included  in  the  Inception  Report  and  presented  in  further 
iterations;  final  approvals  of  the  questionnaires  were  obtained  from  the  corresponding 
contact  points  at  the  EC.  The  final  questionnaires  are  provided  in  Error!  Reference 
source not found. to this report. 

Table 23 Overview – Quantitative surveys  

#  Action 

Type of 
stakeholder 

Population 

Target 
sample 

Number of 
responses 

Response 
rate 

Closing 
date in 
2017 

1  Loan Guarantee 

Beneficiary SMEs 

70,000 

3,870 

356* 

9% 

18 July 

Facility 

2  Enterprise Europe 
Network (EEN) 
3  Enterprise Europe 
Network (EEN) 

4  Cluster 

internationalisation  

5  Tourism (Calls) 

6  Erasmus for Young 
Entrepreneurs 

7  Erasmus for Young 
Entrepreneurs 

210,000 

523** 

136 

EEN member 
organisations 
Beneficiary SMEs  
(EEN clients) 
Intermediary 
agencies and 
cluster 
organisations 
Beneficiary SMEs 
and Intermediary 
agencies / 
organisations  
New Entrepreneurs  2,343 

377 

530 

270 

48% 

17 May 

50,000 

2,334 

5% 

17 May 

136 

48 

35% 

5 May 

389 

162 

41% 

10 May 

2,117 

595 

28% 

5 May 

Host Entrepreneurs  1,800 

1,620 

245 

15% 

5 May 

Source: Technopolis - *Reached via 20 financial intermediaries. **Reached via 92 coordinators. 

4.1.2. 

Sampling strategy and results 

The following paragraphs explain our sampling approach for the quantitative surveys. In 
the  case  of  the  short  surveys  no  sampling  is  needed  as  all  stakeholders  will  be 
approached. 

We  have  taken  careful consideration  of  potential issues  related  to  ‘survey  fatigue’  when 
designing our sampling strategy. 

93 

 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

SO1 - Sampling of the beneficiary SMEs of Loan Guarantee Facility 

The ToR provides a series of good recommendations regarding the sampling strategy for 
the financial instruments.  There are also lessons to learn (in terms of expected response 
rates)  from  the  “Final  Evaluation  of  the  Entrepreneurship  and  Innovation  Programme” 
(2011)87.    Taking  those  elements  into  account,  we  estimated  that  we  would  need  to 
address 70,000 companies to obtain a response of 3,150. 

We  requested  EIF/  DG  Grow  for  access  to  the  full  list  of  LGF  recipient  companies, 
including all contact details necessary for implementing the survey (name, email address, 
telephone number).  

We followed the following steps: 

  We drew a stratified random sample of circa 70,000 companies from the amended list 
and invite them to participate in our (targeted) consultation. Stratification ensured 
the sample has a good spread across countries and sectors (provided this information 
is recorded alongside contact details). 

  To arrive to a sample of 3,870 companies, we needed an (implicit) response rate of 
5.6%, which was virtually the same as the response rate from the EIP. This sample 
size guaranteed a good representation across sectors and countries. 

  A response of 3,150 represented a representative sample, with 95% confidence 

interval and a margin of error of +/- 6%. 

We  monitored  responses  in  real  time  to  make  sure  we  are  achieving  a  good  spread  of 
responses. 

The  Commission  signed  off  on  the  survey  of  stakeholders  of  the  SME  Loan  Guarantee 
Facility on 5 May 2017.  

The  survey  was  translated  into  16  languages  and  released  to  an  estimated  3,870 
beneficiaries via 20 financial intermediaries in 19 countries. The study team constructed 
a random sample of 250 beneficiary companies for those intermediaries that had served 
a greater number of clients. Those intermediaries that had served fewer than 250 clients 
were asked to distribute the survey to all beneficiary clients.  

Via this method, the companies that had received financing supported by an EU-COSME 
guarantee were originally invited to contribute to the study via an online survey between 
19 May and 23 June 2017. Subsequent surveys were launched over the following weeks; 
to  boost  the  number  of  responses  and  facilitate  participation,  the  survey  deadline  was 
first extended to 30 June and then to 7 July 2017. 

The survey received 356 responses, equivalent to a response rate of 9%. 

SO2 - Sampling of the EEN member organisations  

We  approached  EEN  members  through  a  broad  call  that  was  distributed  by  the  EEN 
coordinators.    We  received  access  to  the  full  list  of  EEN  members,  including  all  contact 
details necessary for implementing the survey (name, email address).  

87 http://ec.europa.eu/cip/files/docs/eip-final-evaluation-report_en.pdf  

94 

 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

Enterprise Europe Network member organisations were originally invited to contribute to 
the study via an online survey between 12 April and 3 May 2017. The survey was initially 
released  to  92  coordinators,  and,  to  boost  the  number  of  responses  and  ease  of 
participation,  the  survey  was  subsequently  released  to  431  partner  organisations  on  19 
April. A further reminder e-mail was circulated to both groups on 26 April. 

Over  the  course  of  implementation  of  the  survey,  the  study  team  worked  directly  with 
both  coordinators  and  partners  to  ensure  that  all  relevant  parties  received  the  survey 
invitation.  This  resulted  in  additional  partners  receiving  the  invitation  that  were  not 
initially  included  in  the  data,  and  subsequent  greater  access  to  client  groups  for  the 
second survey. It also ensured that the survey has reached individuals that had replaced 
colleagues that were no longer in relevant positions.  

Survey progress  was briefly interrupted when the EEN members communicated through 
the  SAG  bureau  that  the  timing  of  the  survey  was  a  bit  inconvenient  given  the  holiday 
period, and that they had received word from EEN members that were not in the position 
to send out the SME survey in time. An additional reminder was set out with May 17 as a 
new  end  date  of  the  survey,  providing  the  EEN  members  with  an  additional  opportunity 
to participate in the survey and to send out the SME survey to their clients. 

Overall,  we  received  270  responses,  accounting  for  48%  of  the  target  sample  (the 
highest response rate of the targeted quantitative consultations).  

SO2 - Sampling of the EEN client SMEs  

All  92  coordinators  were  approached  in  the  first  instance,  with  a  request  to  distribute 
details of our survey of client SMEs through the network.   

Working  closely  with  these  coordinators  we  reached  out  to  most  of  EEN’s  523 
organisations,  both  within  and  outside  of  the  EU.  Through  these  organisations,  we  also 
issued  further  requests  to  around  50,000  client  SMEs.  Based  on  our  experience  of  the 
previous EEN evaluation we expected the response rate from EEN member agencies to be 
high.  At  the  same  time,  we  expected  SME  response  rates  to  be  much  lower,  and  we 
aimed for a ~5% response rate from those contacted. 

Prior to the dissemination of both surveys, a letter from the Head of Unit H2 was posted 
on the Enterprise  Europe Network intranet, advising Network members of the upcoming 
consultation,  and  seeking  their  support  to  disseminate  the  SME  survey  to  their  client 
base.  The  time  available  to  answer  the  survey  was  extended  once  to  accommodate  for 
the busy schedules of the stakeholders. 

Enterprise Europe Network client SMEs were also invited to participate in the study via an 
online  survey  questionnaire.  The  survey  of  client  SMEs  was  disseminated  as  a  cascade 
survey,  i.e.  Network  members  invited  to  participate  in  the  first  survey  (above)  were 
asked to support the study in distributing an open web link to their client base. 

As with the survey of  Network members, the survey of  client SMEs was released in two 
phases, the first on 12 April 2017, when the study team wrote to all individual Network 
coordinators  following  their  own  survey  invitation,  and  subsequently,  on  19  April,  when 
the study team wrote to the group of Network partners, following their own invitation to 
participate  in  the  survey  of  Network  members.  We  provided  a  text  that  Network 
members could forward to their own clients to participate in the study.  

The  study  team  worked  closely  with  Network  members  to  ensure  as  high  as  possible 
participation  of  SMEs  in  the  survey.  To  accommodate  national  holidays  in  many 

95 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

countries, the study team further wrote to all 530 coordinators and partners on 26 April 
2017. The survey closed at the end of 17th May. 

Of  the  50,000  SMEs  of  the  target  sample,  we  received  2,334  responses  or  5%.  Though 
rather low, this result met our expectations. 

SO3 - Cluster organisations  

We  addressed  the  136  cluster  organisations  that  are  funded  through  the  Cluster 
Internationalisation  calls.  We  requested  access  to  the  full  list  of  cluster  organisations, 
including  all  contact  details  necessary  for  implementing  the  survey  (name,  email 
address).  

In our survey of cluster organisations in 2014, in the context of the “Evaluation of cluster 
initiatives managed by DG Enterprise and Industry”, we had obtained a response rate of 
circa  8%.  This  time,  we  expected  a  higher  response  rate  as  the  136  clusters  had  only 
recently received support from COSME. We therefore aimed at reaching a response rate 
of at least 25-30%, or 34-40 responses.  

We received 48 answers, or a 35% rate, thus higher than our expectations. 

SO3 – Tourism actions 

We addressed the 377 organisations that were successful in their applications to the open 
calls for proposals in the field of Tourism. The list of these beneficiaries is available in the 
COSME Data Hub.  

Participants and beneficiaries of actions funded through COSME tourism calls were invited 
to contribute to the study via an online survey that took place between 26 April and 10 
May 2017. To boost the response rate, the study team sent a reminder on 3 May 2017. 
The survey was closed on the 10 May 2017. In total, 389 participants and beneficiaries – 
both  intermediary  organisations  and  SMEs  –  across  33  countries  were  invited  to 
contribute to the online survey consultation for this interim evaluation. 

We received 162 answers, or a 41% rate. 

SO4 - EYE Host entrepreneurs 

As the key action funded within the entrepreneurship area, the EYE programme was the 
focus  of  targeted  consultation  surveys.  Specifically,  we  invited  all  Host  and  New 
Entrepreneurs in the programme during the 2014-2016 period to provide information and 
feedback  on  their  participation  in  this  action.  As  was  reported  above,  these  surveys 
received a good response rate from both groups of beneficiaries (with 841 Host and New 
Entrepreneurs responding in total). Analysis of the responses to these questionnaires has 
fed  into  various  parts  of  the  evaluation,  particularly  around  the  effectiveness  and 
efficiency of the programme. Follow-up interviews were also conducted with a number of 
the beneficiaries (as reported below). 

We asked EASME/ DG Grow for access to the full list of Host entrepreneurs, including all 
contact details necessary for implementing the survey (name, email address). 

To avoid survey fatigue, we selected (randomly) a proportion of the total number of Host 
entrepreneurs  involved  in  EYE  in  the  period  2013-2016.  We  reached  a  sample  of  1,620 
Host  entrepreneurs.  Based  on  the  response  rates  obtained  in  the  prior  exercise,  we 

96 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

expected to attain a response from 250 HE as a result. In the end, we received – in line 
with our expectations - 245 answers which make up for 15% of the target sample. 

Host Entrepreneurs were invited to contribute to the study via an online survey that was 
open between 12 April 2017  – 5 May 2017. To boost the response rate, the study team 
sent out a single reminder on 26 April 2017. 

SO4 - EYE New entrepreneurs 

We asked EASME/ DG Grow for access to the full list of Host entrepreneurs, including all 
contact details necessary for implementing the survey (name, email address). 

To avoid survey fatigue, we selected (randomly) a proportion of the total number of New 
entrepreneurs  involved  in  EYE  in  the  period  2013-2016.  We  reached  a  sample  of  2,117 
New  entrepreneurs.  Based  on  the  response  rates  obtained  in  the  prior  exercise  (10%), 
we  expected  to  attain  a  response  from  100  NE  as  a  result.  Of  the  2,117  entrepreneurs 
contacted, we received 595 answers (much higher than our expectations) or a response 
rate of 28%.  

New entrepreneurs were invited to contribute to the study via an online-survey between 
12  April  2017  –  5  May  2017.  To  boost  the  response  rate,  the  study  team  released  a 
single reminder on 26th April 2017.  

4.1.3. 

Robustness of the results 

SO1 – Access to finance 

Overall, the survey received 356 responses, equivalent to a response rate of 9%. This 
represents a higher response rate than recorded in previous exercises such as the 2011 
Final Evaluation of the Entrepreneurship and Innovation Programme (where the response 
rate was approximately 6.3%).88 

Figure 8 below shows a comparison by country of the distribution of the LGF beneficiaries 
as a proportion of the population, of the target sample and the survey population, while 
Figure  9,  displays  the  distribution  of  LGF  beneficiaries  within  sectors  of  activity.  These 
figures  illustrate  the  degree  of  the  representativeness  of  the  survey  population,  and  we 
emphasise that the survey population should not be used to profile all beneficiaries. 

The  survey  did  not  target  all  countries  in  which  there  are  beneficiaries  of  EU-COSME 
guarantees.89  For  the  most  part,  the  shares  of  companies  in  each  country  are  higher 
within  the  survey  population  than  in  the  overall  beneficiary  population,  except  in  Spain 
and Italy.  

Overall,  the  distribution  of  the  survey  responses  is  well  balanced  in  terms  of  country 
and  sector  of  activity  in  comparison  with  the  target  sample.  One  notable  exception  is 
France,  where  despite  our  best  efforts  we  did  not  receive  any  response  as  the  financial 
intermediary did not circulate the survey before the deadline.   

88 See: http://ec.europa.eu/cip/files/docs/eip-final-evaluation-report_en.pdf   

89  The  countries  that  the  survey  covered  account  for  96%  of  the  entire  population  of  beneficiaries  at  the  time  of  the 
consultation (138,102 companies of a total 143,344). The survey did not include intermediaries or companies in the Czech 
Republic, Montenegro, Slovakia or the UK. 

97 

 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

The distribution of sectors is largely consistent between the population and the survey 
respondents.  This  is  particularly  evident  in  the  most  prominent  sectors,  such  as 
‘Wholesale  and  retail  trade;  repair  of  motor  vehicles  and  motorcycles’,  and 
‘Manufacturing’.90 

Figure  8    LGF  beneficiaries:  comparison  of  the  country  distribution  of  the  whole 
population, target sample and survey population 

Source: Technopolis, based on provided beneficiary details and survey data 

Figure  9  LGF  beneficiaries:  comparison  of  the  sector  distribution  of  the  whole 
population, target sample and survey population 

90 There may be slight differences between the beneficiary population and the survey population, as the former is centrally 
recorded, while the survey invited participants to self-report their sector of activity. 

98 

 
 
 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

Source: Technopolis, based on provided beneficiary details and survey data. NB[1]: there is a slight difference 
between the sectors in the provided information (where ‘Wholesale and retail trade’ is combined with ‘Repair of 
motor  vehicles  and  motorcycles’  as  per  the  NACE  classification)  and  the  list  of  sectors  within  the  survey 
questionnaire, where these sectors are presented separately.  The survey data responses have been combined 
for the purposes of this comparison. NB[2]: The reclassification of survey respondents resulted in the addition 
of one sector not originally offered in the survey questionnaire: ‘Water supply; sewerage; waste management 
and remediation activities’. This is not shown in this chart. 

SO2 – Access to market 

Survey of Network Members 

In  total,  the  survey  targeted  531  Network  members:  89  coordinators  and  442  partners 
across  37  countries.  At  the  time  of  reporting  (May  17),  the  survey  has  received  270 
responses from across 35 countries. This represents an overall response rate of 48%. 
The response rates from both coordinator (47%) and partner groups (48%) are roughly 
equal. 

The distribution of the survey population is very similar to the target sample in its two 
main characteristics: country grouping and Network member role. Figure 10 and Figure 
11, below, summarise this. 

Figure 10  Network member survey target sample and respondents, by country  

99 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Breakdown of target sample and survey respondent population - country 

Respondents

Target sample

0%

10%

20%

30%

40%

50%

60%

70%

80%

90% 100%

EU15

EU13

Third country

Breakdown of target sample and survey respondent population - country 

Respondents

Target sample

0%

10%

20%

30%

40%

50%

60%

70%

80%

90% 100%

EU15

EU13

Third country

Breakdown of target sample and survey respondent population - country 

Respondents

Target sample

0%

10%

20%

30%

40%

50%

60%

70%

80%

90% 100%

EU15

EU13

Third country

Source: Technopolis, based on survey data. Base = 270 

Figure 11  Network member survey target sample and respondents, by role 

100 

 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Breakdown of target sample and survey population - roles 

Target sample

Respondents

Target sample

Respondents

Target sample

Respondents

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Partner

Coordinator

Breakdown of target sample and survey population - roles 

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Partner

Coordinator

Breakdown of target sample and survey population - roles 

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Partner

Coordinator

Source: Technopolis, based on survey data. Base = 270 

Survey of Network client SMEs 

The  cascade  approach  meant  that  the  study  team  did  not  have  direct  control  over  the 
number  of  SME  clients  that  are  invited  to  participate  in  the  survey.  We  estimated  the 
target sample for the client SME survey to be 50,000, and set out a target to achieve  a 
response rate of 5% (2,500 client companies). The survey of SME clients received 2,334 

101 

 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

SME  responses  across  34  countries,91  representing  a  response  rate  of  5%  from  the 
estimated target population.  

Over two thirds of respondents (1,639) are located within EU15 Members States, though 
there  are  sufficient  respondents  located  in  EU13  Member  States  (25%,  568 
respondents)  to  analyse  questions  based  on  country  groupings.  The  number  of 
respondents from COSME-participating third countries (4%, 98 respondents) is  too low 
to make meaningful analysis. The distribution of the survey respondent base is shown 
in Figure 12, below. 

Figure 12  Network client SME company survey respondents, by country group 

Please select the country in which your company is located 

4% 

25% 

EU15

EU13

Third country

Other

71% 

Source: Technopolis, based on survey data. Base = 2,309 

SO3 – Favourable environment / Cluster internationalisation 

A  total  of  48  responses  were  received,  equivalent  to  a  35%  response  rate,  which  is 
higher than the desired response rate of 25%. 

The  response  rate  varied  between  the  two  main  groupings:  ‘funded’  and  ‘voluntary’ 
(Figure 13). The response rate for funded organisations (51%) was higher than that for 
voluntary organisations (15%), which is to be expected, given the closer involvement of 
funded organisations with the programme. 

Figure 13  Distribution of population in sample and responses between type of cluster 
organisation 

91 The 34 countries are: 14 of the EU15 countries (Austria, Belgium, Germany, Denmark, Greece, Spain, Finland, France, 
Ireland,  Italy,  Netherlands,  Portugal,  Sweden,  United  Kingdom)  12  of  the  EU13  countries  (Bulgaria,  Czech  Republic, 
Estonia,  Croatia,  Hungary,  Lithuania,  Latvia,  Malta,  Poland,  Romania,  Slovenia,  Slovakia)  and  eight  of  the  11  COSME-
participating third countries (Albania, Armenia, Bosnia and Herzegovina, Moldova, Montenegro, Former Yugoslav Republic 
of Macedonia, Serbia, Turkey) 

102 

 
 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

Source: Technopolis 

Respondents  were  asked  to  characterise  the  main  sector  of  activity  of  their  company. 
Respondents  were  spread  fairly  evenly  across  the  sectors,  with  a  relatively  high 
representation  within  the  information  and  communication  (ICT)  sector  (16.7%).  Nearly 
half  of  the  respondents  placed  the  activities  of  their  company  in  the  ‘Other  (please 
specify)’  category,  which  refers  to  transport  (aerospace  and  rail),  sports,  and  high-tech 
industry. 

SO3 – Favourable environment / Tourism 

The survey received 162 responses from 32 countries92, equivalent to a 41% response 
rate.  Respondents  have  been  split  into  three  main  country  groupings:  EU15  countries 
(those  that  were  Member  States  prior  to  2004),  EU13  countries  (those  that  joined  as 
Member  States  in  2004  and  subsequently),  and  COSME-participating  third  countries.93 
Figure 14,  below,  illustrates  the  high  level  of  alignment  between  the  country  groups 
split  between  the  target  sample  and  respondent  groups,  for  those  for  which  this 
information was available. 

Figure  14    Profile  of  target  sample  and  SO3  tourism  survey  respondents,  by  country 
group 

92  All  EU15  countries  (Austria,  Belgium,  Denmark,  Finland,  France,  Germany,  Greece,  Ireland,  Italy,  Luxembourg, 
Portugal,  Spain,  Sweden,  The  Netherlands,  United  Kingdom),  all  EU13  countries  (Bulgaria,  Croatia,  Cyprus,  Czech 
Republic, Estonia, Hungary, Latvia, Lithuania, Malta, Poland, Romania, Slovakia, Slovenia), and four third countries (Belize, 
Republic of North Macedonia, Iceland, Turkey) 

93 Full list of COSME-participating third countries can be viewed here: https://ec.europa.eu/docsroom/documents/34263 

103 

 
 
 
                                                           
 
Interim Evaluation of the COSME Programme - Methodology Report  

Respondents (N=162)

Target sample (N=393)

Respondents (N=162)

Target sample (N=393)

Respondents (N=162)

Target sample (N=393)

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

EU15

EU13

Third country

Other

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

EU15

EU13

Third country

Other

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

EU15

EU13

Third country

Other

Source: Contact data provided by the Commission, and Technopolis, based on survey data 

SO4 - Entrepreneurship 

New entrepreneurs 

In  total,  2,117  New  Entrepreneurs  from  37  countries  were  invited  to  contribute  to  the 
survey  consultation  for  this  evaluation.  In  the  three  weeks  that  the  survey  was  open, 
595  responses  were  received,  equivalent  to  a  28%  response  rate  (and  higher  than 
the expected response rate of 25%, based on results obtained in prior studies). 

Most  countries  in  the  sample  population  were  represented  by  at  least  one  respondent, 
apart  from  two  EU  countries,  Malta  and  Luxembourg.  However,  it  is  worth  noting  that 
these  countries  only  had  two  contacts  each,  or  0.1%  of  the  sample  population.  The 
numbers for some countries are rather small for the purposes of analysis, so we divided 
them into three groups as explained above. 

Figure 15 shows that the distribution of responses (among those  respondents for which 
the  geographical  location  is  known)  is  very  similar  to  that  of  the  target  sample  when 
considering  the  three  country  groupings:  pre-existing  EU  Member  States  (as  of  2004), 
expansion  countries  (those  joining  the  EU  post-2004),  and  COSME-participating  third 
countries. 

Figure 15  Distribution of population in sample and responses by types of countries 

104 

 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Source: Contact data provided by the Commission, and Technopolis, based on survey data 

Host entrepreneurs 

In  total,  1,620  Host  Entrepreneurs  from  36  countries  were  invited  to  contribute  to  the 
survey  consultation  for  this  evaluation.  In  the  three  weeks  it  was  open,  the  survey 
received  245  responses,  equivalent  to  a  15%  response  rate,  which  is  the  expected 
response rate (based on the results obtained in prior studies). 

Respondents were asked to characterise the main sector of activity of their company. All 
but three of the categories provided were represented by at least one respondent. These 
three  were:  ‘Repair  of  motor  vehicles  and  motorcycles’,  ‘Electricity,  gas,  steam  and  air 
conditioning  supply’,  and  ‘Mining  and  quarrying’.  The  highest  share  of  respondents 
reported  their  main  sector  of  activity  as  ‘Other’,  followed  by  ‘Arts,  entertainment,  and 
recreation’ and ‘Information and communication (ICT)’. 

Figure  16  shows  that  also  in  this  case,  the  distribution  of  responses  (among  those 
respondents  for  which  the  geographical  location  is  known)  is  very  similar  to  that  of  the 
target  sample  when  considering  the  three  country  groupings:  EU13  MS,  EU15  MS  and 
COSME-participating third countries. 

Figure 16  Distribution of population in sample and responses by types of countries  

Source: Contact data provided by the Commission, and Technopolis, based on survey data 

105 

0%10%20%30%40%50%60%70%80%90%100%Sample (N=1620)Respondents (245)Host entrepreneur location distributionEU15EU133rd Cty 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

4.2.  The targeted consultations - qualitative 

4.2.1. 

Overview  

The  qualitative  targeted  consultations  were  used  only  for  two  SOs:  SO1  on  Access  to 
finance  and  SO3  on  Favourable  environment.  As  mentioned  above,  this  type  of  survey 
was implemented where the population was lower than 100. 

Table 24 below presents an overview of the participation in the qualitative surveys. 

Table 24 Overview of the participation in the qualitative surveys 

SO1 – Access to finance 

SO3 – Favourable environment 

LGF intermediaries 
EFG intermediaries 
SME Policy 
SME Policy 
Sectoral 
competitiveness 

E-skills 

Target sample 
21 
12 
28 
127 (according to 
the email list 
received from EC)  
72 
31* 

Number of responses 
16 
4 
15 
23 

7 
8     

Source: Technopolis 

4.2.2. 

Sampling strategy 

SO1  -  Sampling  financial  intermediaries  and  sub-intermediaries  of  Loan  Guarantee 
Facility 

Among the 61 financial intermediaries receiving LGF funds, we decided to approach 21 of 
them.  Five  of  them  (the  financial  intermediaries)  had  been  already  identified  (cat  1 
promotional  banks  and  cat  2  guarantee  societies)  and  other  were  selected  according  to 
their  type  (Promotional  Institution,  Guarantee  Institution,  Commercial  Bank,  Leasing 
Company),  country  of  establishment,  direct/counter  Guarantees  and  risk  category  in 
order to have a wider and representative possible coverage; but also some of them who 
implemented  as  well  CIP  SMEG  and  overall  50%  currently  implementing  also  InnoFin 
SMEG. These gave us insights of the process and leverage effect. 

4.2.3. 

Results for SO1 – Access to finance 

Four qualitative surveys were conducted: 

  qualitative surveys of the LGF and EFG intermediaries  

  qualitative survey of the EFG SMES 

  qualitative survey of the members of European Intermediary Organisations  

A  written  questionnaire  was  sent  via  e-mail  to  21  LGF  intermediaries,  asking  the 
recipients  to  complete  the  questionnaire  within  two  weeks.  Two  reminders  were  sent  to 
LGF intermediaries that did not respond by the fixed deadline. Finally, the team received 
16  completed  questionnaires,  corresponding  to  a  response  rate  of  76%.  One  LGF 
intermediary  explicitly  refused  to  participate,  while  four  intermediaries  did  not  answer. 
Eight  responding  intermediaries  provide  direct  guarantees  for  loans  and  another  eight 
intermediaries provide counter guarantees. 

A  written  survey  of  EFG  intermediaries  was  conducted.  The  team  originally  planned  to 
collect  data  from  five  EFG  intermediaries.  The  responding  EFG  intermediaries  make 
equity  and  equity-related  investments  in  growth-oriented  start-ups  and  SMEs  all  over 

106 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Europe.  The  latter  were  asked  to  cascade  the  survey  to  selected  SMEs94  that  received 
equity  funding  from  the  EFG  instrument  via  the  intermediary.  It  was  planned  that  12 
SMEs were to be contacted by their respective intermediaries. However, only three out of 
five intermediaries completed the written questionnaire for EFG intermediaries. This also 
implied  that  only  a  limited  number  of  SMEs  that  were  beneficiaries  of  the  EFG  (EFG 
SMEs)  could  be  reached.  As  a  result,  the  team  received  only  four  completed 
questionnaires (out of 12) from EFG SMEs. 

Five of the six contacted intermediary organisations were asked by the team to cascade 
the  survey  to  their  members.  Except  for  one  intermediary,  the  European  intermediary 
organisations  contacted  their  members  prior  to  the  interviews  and  collected  information 
on their members’ opinions and views. In two cases, the interviews were even conducted 
in  the  form  of  phone  conferences  in  which  representatives  of  both  the  umbrella 
organisation  and  individual  members  of  the  intermediary  organisations  participated.  In 
this way, the opinions and views expressed during the interviews directly reflect those of 
individual members. 

4.2.4. 

Results for SO3 – Favourable environment 

We launched a total of four ‘qualitative’ surveys related to the actions under SO3. A first 
step in the process was the drafting of the survey questionnaires.  Table 7 below, gives an 
overview of this process, which was rather time consuming due to the number of surveys 
to be covered. 

Table 25 Overview of status of qualitative surveys 

Survey  

Date of draft  

Approval date 

Launch of survey 

SME Policy – SME Envoys 

27.03.2017 

28.03.2017 

21.03.2017  - draft 1; 
27.03.2017 – draft 2 
27.03.2017 

11.04.2017 

11.04.2017 

10.04.2017 
18.04.2017 

10.04.2017 
10.04.2017 

SME Policy – SPR Working 
Group 
E-skills  
Sectoral competitiveness & 
KETS 
The results of these surveys vary substantially. There was a good level of response to the 
survey  in  the  field  of  SME  Policy  (perhaps  due  to  the  fact  that  it  was  sent  out  by  the 
European  Commission),  while  response  rates  were  low  for  e-skills  and  sectoral 
competitiveness.  Even  though  the  contacted  actors  had  participated  in  events  or 
activities of the COSME actions in the field of KETs, e-skills, or sectoral competitiveness, 
some  of  them  mentioned  that  they  were  not  aware  of  the  COSME  programme  nor  had 
they  had  any  interaction  with it.  In  addition,  despite  being  contacted  several  times,  the 
members  of  the  Digital  Coalition  for  Skills  and  Jobs  (to  whom  the  e-skills  survey  was 
targeted)  were  not  responsive  –  only  the  Coalition  members  contacted  for  interviews 
were willing to contribute.  

28.04.2017 
02.05.2017 

Table 26 Overview of responses to SO3 short survey consultations 

Survey 

Actors targeted 

No of targeted actors  No of responses 

SME Policy 

SME Envoys 
SPR Work group members 

Sectoral 
competitiveness 

European stakeholder organisations, 
national stakeholder organisations 

28 
127 (according to the 
email list received from 
EC)  
72 

received 
15 
23 

7 

94 the ones which received EFG investment in the first half of 2016 

107 

 
 
                                                           
Interim Evaluation of the COSME Programme - Methodology Report  

E-skills 

(associations), national and regional 
public authorities   
European stakeholder organisations, 
national stakeholder organisations 
dealing with e-skills 

31* 

8     

Source:  Technopolis  -  *out  of  the  53  e-mail  addresses  received,  only  37  were  valid.  Out  of  them  6  contacts 
were contacted for interviews and 31 for the qualitative survey.  

4.3.  The public consultation 

The objective of the public consultation was to cover all potential relevant stakeholders, 
including those who were not covered by the targeted consultations and interviews, and 
to  collect  feedback  from  them  on  the  COSME  programme  and  its  objectives.  More 
specifically, we expected inputs from: 

  Citizens 

  Companies 

  European business organisations 

  National business organisations 

  Trade unions 

  Non-governmental organisations 

  National public authorities 

  Regional/local public authorities 

  Financial institutions/Finance providers 

  Academia/Research centres 

  Professional consultancies/law firms 

 

Incubators/clusters/innovation support centres 

4.3.1. 

The structure of the public consultation questionnaire  

The  questionnaire  for  the  public  consultation  was  quite  straightforward  as  it  aimed  at 
reaching  all  types  of  stakeholders.  Given  that  the  targeted  consultations  focused 
exclusively  on  the  specific  objectives,  it  was  agreed  to  use  the  public  consultation  to 
collect  data  that  would  allow  a  general  assessment  of  the  evaluation  criteria  for  the 
overall  programme.  The  questionnaire  included  10  questions  focusing  on  all  evaluation 
criteria except for efficiency. 

Because  public  consultations  are  available  in  all  EU  languages,  translation  of  open 
responses  can  become  very  time-consuming,  can  critically  delay  the  treatment  of 
responses,  and  undermines  comparability  between  Member  States.  Therefore,  we 
collected  all  questions  by  means  of  closed  questions  (multiple  choices).  As  it  is  still 
important to provide stakeholders with the opportunity to express themselves in an open 
structure,  we  included  an  open  question  at  the  end  of  the  questionnaire  to  collect 
additional information and all types of comments from the respondents.  

4.3.2. 

The implementation of the public consultation 

The approach of the public consultation followed the consultation process defined by the 
European  Commission  (Better  Regulation  Toolbox/Tool  #50).  Following  an  initial  phone 

108 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

conversation  with  the  client  specifically  regarding  the  public  consultation  in  early 
February 2017, Technopolis Group submitted a draft questionnaire on the 24 March 2017 
and  feedback  was  shared  by  the  European  Commission  on  the  29  March  2017.  A  final 
questionnaire,  resulting  from  an  internal  EC  drafting  and  validation  process,  was 
validated by the end of April 2017.  

As  required  in  the  Terms  of  References,  the  questionnaire  was  available  in  all  EU 
languages. The Commission translation service (DGT) took care of the English version of 
the  consultation  and  further  translations  in  all  other  EU  languages  followed.  The 
European  Commission  implemented  the  questionnaire  for  the  public  consultation  on  the 
“Your Voice in Europe” website and the questionnaire was available in English on 10 May 
2017.  It  remained  accessible  for  12  weeks,  starting  from  the  date  on  which  the  last 
translation was made available online.  

The dissemination of the public consultation fell under the responsibility of the European 
Commission. EASME and the EC’s communication units advertised the public consultation 
on  COSME  and  EASME  websites;  through  EEN  networks  and  relevant  national  contact 
points;  through  other  agencies  and  networks  under  COSME;  and  via  other  EC  social 
media. 

Responses  have  been  monitored  regularly  (brief  update  reported  by  the  EC  every  two 
weeks)  to  identify  the  types  of  stakeholders  (for  example,  stakeholders  in  specific 
countries) from which fewer responses were received and to dedicate efforts to advertise 
the public consultation to them. After the end of the consultation period, the Commission 
forwarded to our team the raw data collected in the original language, in both Word and 
Excel formats. 

Table 27 Overview of the open public consultation process 

Activity 
Phone call with the EC  
Delivery of first draft questionnaire 
Review by the EC 
Validation of final questionnaire 
Editing in EN by EC 
Translation in other EU languages 
Deadline of the consultation 

EC to send inputs to Technopolis Group 
Source: Technopolis 

Timing 
10 February 2017 
24 March 2017 
29 March 2017 
26 April 2017 
26 April – 5 May 2017 
6 May – 12 June 
Launch date: 10 May 
Closing date: 31 August  
Latest results received mid-September (13/09/2017) 

4.3.3. 

Outcomes of the public consultation  

The public consultation was closed on the 31 August 2017. We received the final results 
by mid-September (13/09) 2017. A total of 195 responses were received, as well as 14 
position papers. 

The questionnaire also included questions that allowed us to  profile the respondents to 
the public consultations. 

Half  of  the  respondents  are  active  in  industry/business;  the  second  largest  group  is 
comprised  of  financial  intermediaries  (financial  institutions  or  finance  providers).  The 
respondents  under  ‘others’  included  law  firms,  research  centres,  innovation  centres, 
NGOs and associations; business organisations; public authorities; and citizens (see Figure 
4). Five countries account for two thirds of the answers (France, the UK, Italy, Spain and 
Germany).  

109 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Figure 17 Public consultation - profile of the participants 

4% 

7% 

9% 

11% 

52% 

A company

A financial institution

Others

A national or European business
organisation

A citizen

A national or regional public authority

Source: COSME public consultation, 2017, n=195 

16% 

Two-third  of  the  respondents,  i.e.  88  respondents,  indicated  that  their  organisation  had 
benefitted  from  the  COSME  programme.  Five  respondents  were  involved  in  more  than 
one  action  (e.g.  EEN  as  a  member  organisation  and  beneficiary,  etc.).  The  Loan 
Guarantee Facility (LGF) is, by far, the most recurrent action. 

Table 28 Number of respondents per type of financial support 

Type 
Access to finance 

Action 
Loan Guarantee Facility 

Access to markets 

Entrepreneurship 

Favourable environment 

13 

Equity Facility for Growth 
Enterprise Europe Network (EEN) – EEN member 
organisations 
Enterprise Europe Network (EEN) – beneficiaries 
Erasmus for Young Entrepreneurs – New Entrepreneurs  4 
4 
Erasmus for Young Entrepreneurs – Host 
Entrepreneurs 
Cluster internationalisation 
Tourism (calls) 
Other 
Selected as an expert 
NCP for COSME including financial instruments and 
publishing calls for proposals in all areas / Coordinator 
EEN including extra support for EYE-participation 
COS-DESIGN 

2 
2 
3 

Number of beneficiaries 
55 
29 companies 
16 financial institutions 
10 others 
5 
15 

Source: COSME public consultation (n=88) 

4.3.4. 

Statistical analysis  

The survey responses were analysed using descriptive statistics. 

The  large  number  of  respondents  enabled  us  to  undertake  cross-analyses.  We 
compared  their  views  on  the  relevance  of  COSME  actions  to  their  views  on  the  results 
(and  the  other  way  around)  and  to  the  benefits  they  expect  from  the  programme  (and 

110 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

the other way around). We also compared the expectations of the respondents with the 
areas of improvement they identified (and the other way around).  

5. 

INTERVIEWS 

In  this  chapter,  after  a  brief  overview  (5.1),  we  present  the  results  from  our  interview 
strategy (5.2), here again for each SO. 

5.1.  Overview 

Overall,  our  team  conducted  121  interviews.  Table  29  provides  an  overview  of  the 
interviews conducted for each SO. 

Table 29 Overview of the interviews conducted, by SO 

 SO 

 Interviewed stakeholders 

SO1 – Access to 
finance 

representatives from EU institutions (DG GROW, DG REGIO, DG RTD, DG ECFIN, EIF 
and EIB) 

 Nr 

15 

SO2 – Access to 
market 

intermediary and industry organisations 
EU officials who were responsible for the Network and for other programmes aimed at 
fostering coherence within the Network. 

6 
9 

Network members 
Umbrella organisations were also interviewed so as gain the views of SMEs and 
intermediaries at a higher aggregated level. 

SO3 – 
Favourable 
environment 

EU officials in DG GROW, EASME and other DGs 
Beneficiaries (cluster managers) 
National government representatives 

Umbrella organisations and national experts 

SO4 – 
Entrepreneurship 

Individuals responsible for the design, management and oversight of COSME 
entrepreneurship actions – including European Commission officials, EASME staff and 
individuals within the EYE support office 
Host Entrepreneurs – participants in and beneficiaries of the EYE programme 

New Entrepreneurs - participants in and beneficiaries of the EYE programme 

19 
5 

16 
6 
4 

10 

20 

5 

6 

Source: Technopolis 

5.2.  Results 

5.2.1. 

SO1 – Access to finance 

The team conducted 21 phone interviews with  high level stakeholders. These comprised 
15 interviews with representatives from EU institutions (DG GROW, DG REGIO, DG RTD, 
DG ECFIN, EIF and EIB) and six interviews with intermediary and industry organisations. 
The  rate  of  response  was  equivalent  to  100%,  i.e.  all  relevant  stakeholders  could  be 
reached and interviewed as originally planned. 

See Table 47, Appendix D for the list of interviewees. 

5.2.2. 

SO2 – Access to market 

For  the  evaluation  of  the  Access  to  Markets  actions,  the  interviews  conducted  mainly 
targeted  the  Enterprise  Europe  Network  and  other  programmes/actions  related  to  the 

111 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Network.  We  focused  on  interviewing  EU  officials  who  were  responsible  for  the  Network 
and for other programmes aimed at fostering coherence within the Network. We spoke to 
a  large  number  of  Network  members  in  order  to  gain  deep  knowledge  of  how  the 
Network  works  in  practice.  Finally  umbrella  organisations  were  also  interviewed  so  as 
gain the views of SMEs and intermediaries at a higher aggregated level. 

We  have  been  successful  in  conducting  interviews  during  the  evaluation  despite  the 
coinciding holiday period. Almost all interviews that were foreseen were conducted during 
the time set out in the project planning, and two additional interviews were conducted to 
provide further insight. The list of the 34 interviews is shown in Table 48, Appendix D. 

5.2.3. 

SO3 – Favourable environment 

For  SO3,  we  have  performed  36  interviews  to  cover  the  SME  Policy  and  cluster 
internationalisation  key  actions,  the  case  studies  on  tourism  and  e-skills,  as  well  as  the 
thematic  areas  such  as  KETs  and  sectoral  competitiveness.  Some  interviews  have  been 
performed,  due  to  various  difficulties  in  reaching  the  interviewees.  Bottlenecks 
encountered included the limited availability of some of the interviewees (e.g. in the field 
of  cluster  internationalisation  and  e-skills),  as  well  as  potential  lack  of  interest  or 
suitability  to  contribute.  The  latter  was  especially  the  case  in  the  field  of  sectoral 
competitiveness  and  KETs,  where  stakeholders  are  not  aware  of  COSME  actions  in  this 
field  and  are  reluctant  to  answer  questions.  Where  possible,  replacements  were  found, 
however, in most cases, it was not possible to find a replacement in due time. 

An overview of the number of interviews performed is shown in Table 30. 

Table 30 SO3 – Performed interviews 

EU officials 

DG GROW 
EASME 
Other DGs 

Beneficiaries (intermediaries) 
National governments 
Umbrella organisation & national experts  

Total  

Source: Technopolis 

No. interviews 
conducted 
8 
2 
6 
6 
4 
10 

36 

Table 49,  Appendix  D  lists  the  people  with  whom  interviews  were  conducted  for  all  the 
actions foreseen for the evaluation under the COSME SO3 work stream.  

Table 50 (same appendix) lists the persons with whom interviews were planned initially, 
but  not  performed  by  the  end  of  the  evaluation  due  to  lack  of  availability,  refusal,  or 
other reasons. 

5.2.4. 

SO4 – Entrepreneurship 

The team undertook interviews with three main groups of stakeholders, focusing mainly 
on the key entrepreneurship action (EYE), but also covering many of the other actions in 
the  portfolio.  A  small  number  of  interviews  were  also  undertaken  with  other  relevant 
stakeholders.  The main groups of interviewees were: 

112 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

 

 

Individuals  responsible  for  the  design,  management  and  oversight  of  COSME 
entrepreneurship actions – including European Commission officials, EASME staff and 
individuals within the EYE support office 

Intermediary  organisations  -  which  are  responsible  for  matching,  managing  and 
monitoring EYE relationships between New and Host Entrepreneurs. 

  Host Entrepreneurs – participants in and beneficiaries of the EYE programme 

  New Entrepreneurs - participants in and beneficiaries of the EYE programme 

Table  51,  Appendix  D  lists  the  30  individuals  interviewed  in  relation  to  the  EYE 
programme and other entrepreneurship actions. 

113 

 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Appendix A
realized  

 - COSME programme KPIs, and the extent to which these KPIs have been 

114 

 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Table 31  Legend to Table 32 below 

Page | 115 

COSME's Medium term KPIs' targets for 2017 met. 

KPIs  have  changed.  Even  though  goal  attainment  cannot  be 
measured, new indicators show promising developments. 

Targets not met or no info at all available 

Table 32  Main activities, KPIs, and the extent to which these KPIs have been realised 

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Activities to improve Competitiveness 

Number of simplification 
measures adopted 

The Commission's simplification 
programme was updated in 2010 
and is on track to cut red tape by 
25% in 2012. There were 5 
simplification measures per year 
done until 2010. 

About 7 simplification measures 
per year. 

2014: 27% reduction of the 
burden 

monitoring report 

Number of "fitness" checks 
on quality and value-added 
of activities 

Four "fitness" checks including 
stakeholders were launched in 
2010 for environment, transport, 
employment and industrial 
policies. Feedback included 
comments on legislation and 
value-added of activities. 

The feedback approach with 
"fitness" checks will be extended 
to other policies and lead to 
simplifications impacting 
positively on industry. Up to 
twelve "fitness" checks are 
foreseen, with the objective of 
better regulation. 

2014-2016: 15 regulation impact 
assessments 
2014: 8 
2015: 4 
2016: 3 

Data provided by DG 
GROW 

115 

 
 
 
 
 
 
 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Page | 116 

Level of adoption by 
companies of European 
sustainable production and 
product tools, including 
EMAS, eco-label, and eco-
design 

Approximately 35,000 ISO 14001 
EMS certifications and 4,500 
EMAS registration, 18,000 
licences for the EU Ecolabel 

Significant number of companies 
monitor their performance, apply 
environmental management 
systems and achieve 
improvement in resource 
productivity and environmental 
performance. Significant part of 
production are resource efficient 
and environmentally friendly 
products 

Developing SME policy 

Number of Member States 
using SME test 

Number of Member States using 
SME test: 15 MS 

Number of Member States using 
SME test: 21 MS 

(2005-)2015: 39,340 EMAS/ISO 
14001 certifications 

http://ec.europa.e
u/environment/em
as/emas_registrati
ons/statistics_grap
hs_en.htm 

2015: "Of the 24 responding 
countries, 20 countries reported 
that they already had an SME-
test in place" 

EU MSs reporting 
about their SME test 
http://ec.europa.eu/
DocsRoom/document
s/19261 

Increased EU-wide publicity 
of the European Enterprise 
Awards with media 
publications/clippings in all 
Member States 

 Number of media 
publications/clippings in all 
Member States: 60 in 2010 

Number of media 
publications/clippings in all 
Member States: 80 

2014: 6 
2015: 15 

Meltwater (maybe 
they have the data 
internally) 

Reduction of start-up time 
and complexity for new 
enterprises 

Reduction of start-up time: 7 
working days 

Reduction of start-up time: 5 
working days 

In 2015, the average time to 
start a private limited company 
in the EU was 3.4 days and the 
cost was EUR 315. The reduction 
in average time is mostly due to 
simplifications implemented in 
Austria, Bulgaria and Sweden. 
The slight augmentation of 
average cost is mostly due to 
increasing costs in Hungary 

http://ec.europa.eu/
growth/smes/promot
ing-
entrepreneurship/ad
vice-
opportunities/start-
up-procedures_en 

116 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Page | 117 

New business concepts 

Number of new 
products/services in the 
market 

So far, this activity was restricted 
to analytical work of limited scale. 

Target for the cumulative number 
of new products/services to be 5 
in 2017 (increasing to 15 in 2018 
and 25 in 2019). 

Design-based consumer goods? 
2015: EUR 11.2 Mio budget, 27 
proposals, 4 awarded projects 
2016: EUR 4 Mio, 26 proposals, 6 
awarded projects 

Call COSME Design 
Based Consumer 
Goods - Main 
outcomes & 10 
success stories 
(EASME 
presentation) 

Level of additional exports 
and corresponding monetary 
amounts 

Feedback from stakeholders 
on quality and value-added 
of activities 

As for exports, no expected 
impact in 2017 yet. The share of 
exports of the first generation of 
participating SMEs will appear in 
2018 with a target increase of 
20%. 

N/A 

N/A 

At least 70% of SMEs 
participating in 2014 shall express 
a positive impact on their turn 
over in a survey done end 2017 

Design-based consumer goods? 
2015: "The stage of 
implementation does not allow 
measuring other relevant 
indicators" 

monitoring report 

Tourism 

Number of applications to 
funding 

Number of applications to funding 
(to all calls for proposals) in total: 
around 75 per year (average for 
2011) 

Number of applications to funding 
(to all calls for proposals) in total: 
more than 100 per year 

2014: n/a 
2015: 216 

Percentage of SMEs (and 
trend) in applications for 
tourism-related funding 
opportunities 

Up to date, no calls for proposals 
were directly addressed to SMEs  

30% of calls for proposals directly 
addressed to/involving  SMEs in 
partnerships 

Most of calls for proposals 
(except ad-hoc grants) required 
the involvement of SMEs in the 
projects consortia.  

SMEs account for around 150 out 
of 377 organisations which were 

monitoring report 

EASME/ COSME data 
hub  

117 

 
 
  
  
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Page | 118 

Number of destinations 
adopting the sustainable 
tourism development 
models promoted by the 
European Destinations of 
Excellence 

Number of European Destinations 
of Excellence awarded in total 98 
(on average 20 per year – in 
2007- 10, in 2008-20, in 2009-
22, in 2010-25, in 2011-21, 2013 
- 19) 

158 winners and up to 500 
runners-up destinations adopting 
the sustainable tourism 
development models promoted by 
the European Destinations of 
Excellence initiative. 

Support for entrepreneurship 

Feedback on the public 
perception of 
entrepreneurship (% of EU 
citizens that would like to be 
self-employed as measured 
by Eurobarometer) 

Figures from 2007 and 2009 are 
stable at 45% 

Increase of EU citizens that would 
like to be self-employed to 50% 

118 

beneficiaries of the open calls for 
proposals in the field of tourism, 
which stand for roughly 40% of 
all beneficiaries. 

(Out of the SMEs beneficiaries 
around 100 companies were 
active in the tourism sector and 
additional 50 private enterprises 
were active in other sectors) 

2014: new 18 destinations 
selected 
2015: 13 projects on promotion 
of the EDEN destinations 
awarded 
2016: new 17 destinations 
selected 

2017: up to 17 projects on 

promotion of the EDEN 

destinations to be awarded 

Most recent Eurobarometer on 
entrepreneurship is 2012 
2012: 37% of EU27 would prefer 
to be self-employed 
32% of EU27 find it desirable 
(fairly + very) to become self-
employed within the next five 
years 
53% of EU27 have a broadly 
favourable opinion of 
entrepreneurs 

EASME 

Flash Eurobarometer 
354 (2012) (volume 
AA) 
http://data.europa.e
u/euodp/en/data/dat
aset/S1024_354  

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Page | 119 

Number of states 
implementing 
entrepreneurship solutions 
developed at EU level: 22 
(2010) 

Number of nationally run 
programmes available to SMEs 
from other MS: 5 

Number of nationally run 
programmes available to SMEs 
from other MS: 10 

Entrepreneurship Education? 
2015: "to early to assess how 
many countries participated in 
the activities of the network" 

monitoring report 

Number of simplification 
measures adopted for SMEs 

5 simplification measures per 
year (2010). 

About 7 simplification measures 
per year 

No information available 

N/A 

Financial Instruments for growth 

Number of firms receiving 
loan (credit) guarantees and 
value of lending 

Proposed instruments are not yet 
launched and different from 
current instruments, so data from 
current instruments may not be 
comparable 

Number of firms receiving loan 
(credit) guarantees (+/-95 000) 
and value of lending (+/- EUR 
10.7 billon) 

Number of VC-backed firms 
and value of investments (of 
which cross border deals) 

Number of VC-backed firms: (+/- 
180) and value of investments 
(+/- EUR 220m) 

Number of partnership 
agreements signed 

Partnership agreements signed: 
1.950 (2010) 

Partnership agreements signed: 
3.000/year 

Enterprise Europe Network 

2014: 3 operations signed, EUR 
851 Mio of financing to around 
13,000 SMEs (leverage: 1 to 10) 
2015 (aggregate): EUR 1,292 
Mio of financing to more than 
51,000 SMEs 
2015 (expected): 24 operations, 
EUR 6,200 Mio of financing to 
around 95,000 SMEs (leverage: 1 
to 30) 

2014 (expected for the lifetime of 
the programme): EUR 2,6 to 4 
billion, risk capital to 362 to 544 
firms 
2015: 5 operations, EUR 395 
Mio, risk capital to 53 SMEs 

monitoring report 

monitoring report 

2013-2014: 5,273 SMEs 
2015-2016: 6.223 SMEs 

(2 years periods) 

monitoring report 

119 

 
 
  
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

Page | 120 

Increased recognized 
Network brand and brand 
Culture (e.g. brand 
awareness among SME 
population) 

Not measured yet 

30% of SMES reached 

Clients satisfaction rate (% 
SMEs stating satisfaction, 
added-value of specific 
service) 

Clients satisfaction rate (% SMEs 
stating satisfaction, added-value 
of specific service): 78% 

Clients satisfaction rate (% SMEs 
stating satisfaction, added-value 
of specific service): >80% 

2014: 411,000 SMEs involved in 
promotion and information local 
events 
2015: 8% recognition among 
SME population (Eurobarometer 
survey n°421) 

2015: 10,5 mio SMEs reached via 
(digital) information services 

2014: 85% 
2016 : 86% of respondents 
declared to be satisfied or very 
satisfied with EEN services 

93% would recommend it to 
others 

monitoring report 

monitoring report 

Number of SMEs receiving 
support services 

Number of SMEs receiving 
support services: 435.000 (2010) 

Number of SMEs receiving support 
services 250.000/year (revised 
indicator) 

2014: 132,668 SMEs 
2015-2016: 254.057 SMEs/Y 

monitoring report 

Number of SMEs 
participating in brokerage 
events and company 
missions 

Share (%) of SMEs involved 
in international activities 
(exports, imports, FDI and 
other activities) outside the 
EU) 

Number of SMEs participating in 
brokerage events and company 
missions: 45.000 (2010) 

Number of SMEs participating in 
brokerage events and company 
missions: 60.000/year 

2013-2014: 50,719 SMEs 
2015-2016: 45.953 SMEs with 
156.408 meetings 

monitoring report 

SME business support in markets outside the EU 

13 % (2009) 

17 % (2017) 

120 

2015: 
- Exported: 20% 
- Imported: 19% 
- Subcontractor based abroad: 
7% 
- Worked as a subcontractor: 5% 
- Worked for R&D with a partner 
abroad: 4% 
- Invested in a company abroad: 

Flash Eurobarometer 
421 (2015) 

 
 
Interim Evaluation of the COSME Programme - Methodology Report  

Result indicator  

Situation in 2011 

COSME's Medium term KPIs 
for 2017 

Did COSME meet the KPI 
targets? 

 Source 

2% 

Page | 121 

International Industrial Cooperation 

It is estimated that in regulatory 
co-operation with main trading 
partners (US, Japan, China, 
Brazil, Russia, Canada, India) 
there is an average of 2 relevant 
areas of significant alignment of 
technical regulations 

It is estimated that on average in 
the three policy region (candidate 
countries region, neighbourhood 
East and neighbourhood MED) of 
the 10 policy areas of the SBA at 
least 3 of those have been 
regulated in these countries. 

3 relevant areas of significant 
alignment of technical regulations 
with main trading partners (US, 
Japan, China, Brazil, Russia, 
Canada, India) (2017) 

5 policy areas of the SBA in the 
three policy region 
(candidate countries region, 
neighbourhood East and 
neighbourhood MED) (2017) 

Number of cases of 
improved alignment 
between EU and third 
countries’ regulations for 
industrial products 

Number of areas and good 
practices of the EU Small 
Business Act which have 
been introduced in 
neighbourhood and 
candidate countries 

Source: Techonopolis

No information available 

N/A 

No information available 

N/A 

121 

 
 
Appendix B

 -  Statistical data analyses results 

 Analysis of the geographical distribution 

B.1  
The action lines considered in the analysis, for which data on final  beneficiaries are 
available, are: 

  LGF 

  EFG 

  EEN 

  EYE 

  Cluster Internationalisation 

  Tourism 

Table 33 below presents the penetration rate by country for the action line LGF. The 
penetration  rate  measures  the  percentage  of  SMEs,  employees  and  SME  turnover 
reached by LGF compared to total number of SMEs, employees and SME turnover in 
the country. 

Table 33 Country penetration rates of LGF guarantee 2014-2016 

LGF SMEs (all) 
Country 

Total 
number 
of SMEs 

Total value of 
SME Turnover 
(First inclusion) 
(in EUR) 

Averag
e age 

Penetration rates… based on: 
number 
of 
SMEs 

SME 
Turnover 

number 
of 
employe
es 

Austria 
Belgium 
Bulgaria 
Czech Republic 
Denmark 
Estonia 
France 
Germany 
Greece 
Hungary 
Italy 
Latvia 
Lithuania 
Montenegro 
Netherlands 
Poland 
Romania 
Slovakia 
Slovenia 
Spain 
United Kingdom 
Total 
Source: Technopolis, based on EIF monitoring data at end of 2016 and National statistics 2016 (2017) 

907,520,155 
955,471,071 
949,484,069 
2,459,727,894 
6,404,467 
576,987,811 
8,577,346,400 
3,568,813,371 
523,598,067 
741,963,920 
19,111,421,957 
111,387,715 
253,300,460 
191,344,923 
50,546,318 
2,325,042,247 
408,234,103 
31,315,673 
723,330,826 
7,287,511,069 
2,204,655,663 
51,965,408,179 

1,176 
471 
660 
1,812 
209 
255 
51,047 
8,840 
322 
1,475 
28,518 
67 
71 
521 
440 
3,351 
642 
146 
392 
40,166 
2,763 
143,344 

0.40% 
0.10% 
0.20% 
0.20% 
0.10% 
0.40% 
1.60% 
0.40% 
0.00% 
0.30% 
0.80% 
0.10% 
0.00% 
1.60% 
0.00% 
0.20% 
0.10% 
0.00% 
0.30% 
1.70% 
0.20% 
0.60% 

0.40% 
0.20% 
0.90% 
0.70% 
0.10% 
1.30% 
1.60% 
0.10% 
0.20% 
0.60% 
1.30% 
0.30% 
0.50% 
3.80% 
0.00% 
0.40% 
0.30% 
0.00% 
2.10% 
1.30% 
0.20% 
0.70% 

7.6 
5.1 
9.1 
11.3 
6.9 
10.4 
9.5 
2.3 
16.1 
11.5 
13.3 
12.8 
13.2 
10 
5.3 
7.4 
10.8 
8.7 
15.6 
6.8 
8.6 
9 

0.00% 
0.00% 
0.01% 
0.01% 
0.00% 
0.01% 
0.00% 
0.00% 
0.00% 
0.00% 
0.00% 
0.00% 
0.00% 
0.02% 
0.00% 
0.00% 
0.00% 
0.00% 
0.01% 
0.00% 
0.00% 
0.00% 

Total 
number of 
employees 
(First 
Inclusion) 
8,110 
4,062 
12,797 
16,371 
840 
4,370 
155,539 
25,475 
3,972 
10,290 
142,168 
1,453 
3,770 
3,563 
916 
24,381 
6,665 
521 
8,767 
102,973 
19,041 
556,044 

Table  34  presents,  for  each  action  line  analysed,  the  shares  of  beneficiaries 
belonging  to  a  country  aggregation  (EU15,  EU13,  COSME  3rd  countries  and  Other 
countries) over the total number of beneficiaries reached by that thematic area. 

Table  34  Geographical  distribution  -  shares  by  country  aggregation  of  number  of 
beneficiaries for 2014-2016 

Action line 

EU15 

EU13 

3rd countries 

Other 

122 

 
 
 
Action line 

Access to finance / LGF 

Access to finance / EFG 

EEN 

EYE 

Cluster internationalisation  

Tourism grants 

Grand total 

EU15 

80.9% 

0.01% 

11.1% 

2.4% 

5.4% 

0.2% 

100% 

EU13 

49.3% 

3rd countries 

Other 

18.6% 

40.7% 

73.5% 

100.0% 

7.6% 

1.8% 

0.5% 

7.7% 

0.2% 

100% 

100% 

100% 

Source:  for  LGF  and  EFG:  EIF  quarterly  report  Q4/2016;  for  EEN  and  EYE:  databases  from  COSME;  for 
Cluster  Internationalisation:  European  Cluster  Collaboration  Platform.  For  all  the  other  action  lines: 
datahub. Note: the datahub only contains information on calls for proposal. 

Table 35 reports the estimated number of beneficiaries reached in each EU country 
by the different action lines. 

Table  35  Geographical  distribution  –  estimated  number  of  final  recipients  by 
country in EU for 2014-2016  

Cluster 
Internationalisation 
250 

Tourism 
grants 
12 

Country  Access to 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

EL 

ES 

FI 

FR 

HR 

HU 

IE 

IT 

LT 

LU 

LV 

MT 

NL 

PL 

PT 

RO 

SE 

SI 

finance / LGF 
1,176 

471 

660 

0 

1,812 

8,840 

209 

255 

322 

40,166 

0 

51,047 

0 

1,475 

0 

28,518 

71 

0 

67 

0 

440 

3,351 

0 

642 

0 

392 

Access to finance 
/ EFG 

EEN 

EYE 

351 

590 

553 

84 

863 

125 

149 

60 

84 

82 

586 

114 

0 

52 

2 

2 

4 

1 

1 

1 

2,630 

345 

1,050 

198 

177 

815 

3,174 

231 

56 

30 

227 

951 

36 

914 

0 

112 

1,375 

140 

2,449 

174 

2,171 

293 

692 

219 

76 

84 

66 

0 

43 

0 

2,011 

1,025 

879 

135 

25 

9 

74 

10 

199 

246 

142 

253 

75 

126 

0 

0 

0 

440 

88 

254 

0 

480 

0 

406 

19 

351 

67 

1,275 

2,013 

450 

1,025 

558 

413 

123 

19 

8 

2 

3 

9 

4 

3 

13 

57 

2 

21 

9 

9 

7 

80 

5 

1 

8 

6 

10 

2 

11 

5 

1 

25 

 
  
  
  
  
  
  
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Country  Access to 

SK 

UK 

finance / LGF 
146 

2,763 

Total 

142,823 

1 

12 

Access to finance 
/ EFG 

EEN 

EYE 

Cluster 
Internationalisation 
0 

Tourism 
grants 
2 

388 

3,392 

114 

393 

301 

25,687 

5,346 

9,274 

18 

352 

Source: for LGF and EFG :  EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; for 
Cluster  Internationalisation:  European  Cluster  Collaboration  Platform.  For  all  the  other  action  lines: 
datahub. Note: the datahub only contains information on calls for proposal. 

Table  36  provides  the  percentages  of  beneficiaries  reached  by  each  action  line, 
weighted by the number of SMEs in each country. 

Table 36 Geographical distribution – weighted shares of participants by country in 
EU for 2014-2016 

Country  Access to 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

EL 

ES 

FI 

FR 

HR 

HU 

IE 

IT 

LT 

LU 

LV 

MT 

NL 

PL 

PT 

RO 

SE 

SI 

SK 

UK 

finance / LGF 
0.4% 

0.1% 

0.2% 

0.0% 

0.2% 

0.4% 

0.1% 

0.4% 

0.0% 

1.7% 

0.0% 

1.6% 

0.0% 

0.3% 

0.0% 

0.8% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.2% 

0.0% 

0.1% 

0.0% 

0.3% 

0.0% 

0.2% 

Total 

0.6% 

Access to 
finance / EFG 
0.0% 

EEN 

EYE 

0.1% 

0.0% 

Cluster 
Internationalisation 
0.1% 

Tourism 
grants 
0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.1% 

0.0% 

0.1% 

0.2% 

0.0% 

0.0% 

0.2% 

0.2% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.4% 

0.3% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.1% 

0.1% 

0.0% 

0.1% 

0.1% 

0.0% 

0.1% 

0.2% 

0.1% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.2% 

0.1% 

0.0% 

0.1% 

0.0% 

0.0% 

0.4% 

0.1% 

0.0% 

0.3% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.2% 

0.1% 

0.0% 

0.1% 

0.0% 

0.1% 

0.3% 

0.1% 

0.0% 

0.1% 

0.0% 

0.0% 

0.2% 

0.0% 

0.0% 

0.1% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

Source: for LGF and EFG :  EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; for 
Cluster  Internationalisation:  European  Cluster  Collaboration  Platform.  For  all  the  other  action  lines: 
datahub. Note: the datahub only contains information on calls for proposal. 

124 

 
 
Table 37  shows  the  estimated  shares  of  cumulative  budget  allocated  for  the  action 
lines analysed, over the cumulative budget allocated in each group of countries. The 
cumulative budget is the budget allocated for each action line from January 2014 to 
December 2016. 

The  action  line  EYE  is  not  analysed  because  the  amount  of  money  received  by  the 
final  beneficiaries  (the  young  entrepreneurs)  depends  on  the  country  in  which  they 
work. 

Table  37  Geographical  distribution  -  shares  by  country  aggregation  of  cumulative 
budget for 2014-2016  

Action line 

Access to finance / LGF 

Access to finance / EFG 

Business mgt capacity (EEN) 

Entrepreneurship 

EU15 

96.5% 

1.5% 

1.7% 

0.2% 

EU13 

96.3% 

3.1% 

0.4% 

Tourism 

Grand total 

0.1% 

0.3% 

100.0% 

100.0% 

3rd countries 

58.5% 

38.3% 

1.6% 

1.1% 

100.0% 

Source: for LGF and EFG : EIF quarterly report Q4/2016; for EEN and EYE: databases from COSME; For all 
the other action lines: datahub. Note: the datahub only contains information on calls for proposal. 

Table  38  presents  the  total  overview  of  budget  allocated  to  final  beneficiaries 
through the four action lines LGF, EFG, Tourism and EEN, between 2014 and 2016. 

Table 38 Geographical distribution (money allocated to final beneficiaries in EUR k 
for 2014-2016) 

Country 
AL 

AM 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

EL 

ES 

FI 

FR 

HR 

HU 

IE 

IS 

IT 

LT 

LGF* 
 -      

- 

216,065 

81,172 

91605 

 -      

199,655 

EFG* 
 -      

- 

 -      

 -      

 -      

 -      

 -      

552,956 

36,011 

17,968 

73,580 

34,189 

 -      

 -      

11,237 

Tourism 
 -      

- 

490 

583 

163 

103 

102 

322 

113 

130 

425 

EEN 
233 

144 

3,960 

3,719 

2,112 

649 

2,736 

Grand total 
233 

144 

220,515 

85,474 

93,880 

752 

202,493 

22,676 

611,965 

2,074 

510 

0 

20,155 

74,220 

45,851 

1,011,408 

 -      

 1 421    

14,125 

10,255,33 

 -      

 -      

1,274,179 

81,52 

 -      

49,878 

 -      

 -      

 -      

 -      

1,000 

 -      

55 

668 

303 

345 

225 

53 

1,784 

1,839 

15,821 

1,298,820 

1,157 

2,711 

0 

527 

1,460 

52,934 

1,225 

580 

1,532,541 

1,250 

 2 197    

17,373 

1,551,164 

 12,226    

 -      

201 

898 

1,099 

125 

 
  
  
Country 
LU 

LV 

ME 

MK 

MT 

NL 

PL 

PT 

RO 

RS 

SE 

SI 

SK 

TR 

UK 

LGF* 
 -      

6,177 

11,012 

 -      

 -      

EFG* 
 -      

 -      

 -      

 -      

 -      

17,318 

3,000 

138,928 

 -      

42,545 

 -      

 -      

106,561 

3,318 

 -      

 -      

 -      

 -      

 -      

 -      

 -      

 -      

 -      

74,050 

3,364 

Tourism 
16 

169 

 -      

43 

205 

370 

99 

204 

86 

 -      

38 

590 

28 

85 

439 

EEN 
705 

649 

179 

273 

0 

0 

8,285 

2,844 

2,177 

482 

2,959 

1,161 

1,170 

5,578 

0 

Grand total 
721 

6,995 

11,191 

316 

205 

20,688 

147,312 

3,048 

44,808 

482 

2,997 

108,312 

4,516 

5,663 

77,853 

Grand Total 

5,547,173 

64,014 

6,653 

119,671 

5,725,443 

Source:  for LGF and EFG - EIF quarterly report Q4/2016; for Entrepreneurship education - EYE database; 
for all the other action lines - datahub. Note: the datahub only contains information on calls for proposal. 
* The figures presented in the table for LGF and EFG refers to investments into eligible final beneficiaries 
(financing transactions). 

 -  Analysis of the sectoral involvement 

B.2  
This  section  provides  the  distribution  by  sector  of  the  number  of  beneficiaries 
reached  and  budget  allocated  to  final  beneficiaries,  for  the  action  lines  LGF,  EFG, 
EEN, EYE, Cluster Internationalisation and Tourism grants.  

Table  39  and  Table  40  present  the  breakdown  of,  respectively,  the  estimated 
number and the shares of final beneficiaries reached, by the principal NACE sectors. 

Table 39 Sectoral involvement (estimated shares of cumulative budget 2014-2016) 

Sector 

Description 

Access to 
finance / 
LGF 

EEN 

EYE 

Cluster 
internationalisation  

Tourism 
grants 

Grand 
total 

A 

C 

D 

E 

F 

G 

H 

Agriculture, 
forestry and 
fishing 
Manufacturing 

5,874 

16,564 

128 

487 

18,057 

36,830 

Electricity, gas, 
steam and air 
conditioning 
supply 
Water supply; 
sewerage; 
waste 
management 
and 
remediation 
activities 
Construction 

Wholesale and 
retail trade; 
repair of motor 
vehicles and 
motorcycles 
Transporting 
and storage 

Access 
to 
finance 
/ EFG 
1 

2 

1 

- 

 - 

3 

508 

172 

1,291 

8,276 

625 

3,076 

2,132 

69 

1,444 

- 

26 

349 

 - 

 - 

647 

190 

133 

 - 

8,907 

 - 

 - 

42 

373 

126 

2 

- 

- 

- 

- 

- 

5 

7,848 

28,543 

3,774 

862 

18,894 

36,966 

9,327 

 
Sector 

Description 

I 

J 

M 

N 

N79 

Q 

S 

Accommodation 
and food 
service 
activities 
Information 
and 
communication 
Professional, 
scientific and 
technical 
activities 
Administrative 
and support 
service 
activities 
Tourism 
services 
Human health 
and social work 
activities 
Other services 
activities 

Access to 
finance / 
LGF 

14,666 

Access 
to 
finance 
/ EFG 
 - 

 - 

381 

 - 

EEN 

EYE 

Cluster 
internationalisation  

Tourism 
grants 

Grand 
total 

2,881 

11,626 

1 

2 

4,255 

647 

2,196 

2,540 

1,303 

 - 

11 

2 

42 

15,058 

9,982 

15,513 

5,751 

 - 

4,350 

 - 

 - 

114 

10,215 

- 

4,221 

9,697 

- 

 - 

2 

- 

- 

- 

- 

2,913 

150 

264 

5,235 

 - 

499 

1,366 

91 

130 

23 

- 

29 

130 

7,571 

14,934 

9,640 

Other 

Other sectors 

7,655 

TOTAL 

143,344 

12 

30,708  5,561  9,274 

358 

189,257 

Source: for LGF and EFG : EIF quarterly report Q4/2016; for EYE and EEN: data from EASME; for all the 
other action lines: datahub. Note: the datahub only contains information on calls for proposal. Note: the 
sectors  for  the  final  beneficiaries  of  the  calls  for  proposals  have  been  attributed  on  the  basis  of  the 
information available; data for the sectoral involvement in LGF includes also Montenegro. 

Table 40 Sectoral involvement (shares of final beneficiaries 2014-2016) 

Sector  Description 

Access to 
finance / LGF 

Access 
to 
finance 
/ EFG 
5% 

EEN  EYE 

Cluster 
internationalisati
on  

Tourism 
grants 

Grand 
total 

2% 

3% 

14% 

1% 

4% 

A 

C 

D 

E 

F 

G 

H 

I 

J 

M 

N 

Agriculture, 
forestry and 
fishing 
Manufacturing 

Electricity, gas, 
steam and air 
conditioning 
supply 
Water supply; 
sewerage; waste 
management and 
remediation 
activities 
Construction 

Wholesale and 
retail trade; 
repair of motor 
vehicles and 
motorcycles 
Transporting and 
storage 

Accommodation 
and food service 
activities 
Information and 
communication 
Professional, 
scientific and 
technical 
activities 
Administrative 
and support 
service activities 

4% 

0% 

13% 

26% 

6% 

10% 

2% 

8% 

4% 

12% 

16% 

27% 

11% 

33% 

0% 

11% 

7% 

1% 

16% 

0% 

4% 

12% 

2% 

16% 

2% 

18% 

3% 

0% 

8% 

16% 

1% 

4% 

1% 

4% 

7% 

3% 

7% 

21% 

14% 

12% 

24% 

1% 

7% 

16% 

8% 

23% 

12% 

9% 

14% 

32% 

6% 

127 

 
  
  
  
 
  
  
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Sector  Description 

Access to 
finance / LGF 

Access 
to 
finance 
/ EFG 

EEN  EYE 

Cluster 
internationalisati
on  

Tourism 
grants 

Grand 
total 

N79 

Tourism services 

Q 

S 

Human health 
and social work 
activities 
Other services 
activities 

3% 

7% 

0% 

3% 

3% 

16% 

17% 

0% 

36% 

6% 

Other  Other sectors 

6% 

31%  

2% 

25% 

1% 

8% 

0% 

5% 

8% 

4% 

TOTAL 

100% 

100% 

100
% 

100% 

100% 

100% 

100% 

Source: for LGF and EFG -  EIF quarterly report Q4/2016; for EYE and EEN: data from EASME; for all the 
other action lines - datahub. 

Table 41 presents the  amount of budget (in million Euros) allocated by each action 
line to beneficiaries, distributed by NACE sector, while Table 42 shows the shares of 
budget allocated in each sector for each of the action lines analysed. 

Table 41 Sectoral involvement (budgets in EUR m) 

Sector 

LGF 

EFG 

EYE 

Tourism 

Grand Total 

A - Agriculture, forestry and fishing  327.2 

C - Manufacturing 

F - Construction 

1,053 

584.3 

4 

3 

G - Wholesale and retail trade; 
repair of motor vehicles and 
motorcycles 

1,401.40 

35.9 

H - Transporting and storage 

389.4 

I - Accommodation and food 
service activities 
M - Professional, scientific and 
technical activities 

447.4 

312.7 

14.8 

N - Administrative and support 
service activities 

180.8 

N79 - Tourism services 

Q - Human health and social work 
activities 

159.4 

R - Arts, entertainment and 
recreation 
S - Other services activities 

84 

244.7 

6.7 

Other sectors 

362.8 

12.6 

0 

1 

0 

0 

0 

0 

3 

0 

0.1 

0 

331.2 

1,057 

584.3 

1,437.3 

0.1 

389.5 

0.2 

1.3 

3 

3.5 

0.5 

0.8 

447.6 

331.8 

183.8 

3.5 

159.9 

84.9 

251.4 

375.4 

Grand total 

5,547.1 

77 

4.1 

9.4 

5,637.6 

Source:  for  LGF  and  EFG:  EIF  quarterly  report  Q4/2016;  for  EYE:  data  from  EASME;  for  all  the  other 
action lines: datahub. Note: the datahub only contains information on calls for proposal. Note: the sectors 
for  the  final  beneficiaries of  the  calls  for  proposals  have been  attributed  on  the basis  of  the  information 
available. 

128 

 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Table 42 Sectoral involvement (number of final beneficiaries) 

Sector 

A - Agriculture, forestry and 
fishing 

LGF 

EFG 

5.9% 

5.2% 

EYE 

0.0% 

C - Manufacturing 

19.0% 

3.9% 

24.4% 

Tourism 

Grand total 

F - Construction 

10.5% 

0.0% 

G - Wholesale and retail trade; 
repair of motor vehicles and 
motorcycles 

25.3% 

46.6% 

H - Transporting and storage 

7.0% 

0.0% 

0.0% 

0.0% 

0.0% 

0.0% 

1.1% 

2.1% 

0.0% 

0.0% 

0.0% 

0.0% 

5.9% 

18.7% 

10.4% 

25.5% 

6.9% 

7.9% 

5.9% 

I - Accommodation and food 
service activities 

M - Professional, scientific and 
technical activities 

N - Administrative and support 
service activities 

8.1% 

0.0% 

5.6% 

19.2% 

73.2% 

13.8% 

3.3% 

0.0% 

0.0% 

31.9% 

3.3% 

N79 - Tourism services 

0.0% 

0.0% 

Q - Human health and social work 
activities 

2.9% 

0.0% 

R - Arts, entertainment and 
recreation 

1.5% 

0.0% 

S - Other services activities 

4.4% 

8.7% 

0.0% 

0.0% 

2.4% 

0.0% 

0.0% 

37.2% 

5.3% 

8.5% 

0.0% 

0.0% 

0.1% 

2.8% 

1.5% 

4.5% 

6.7% 

Other sectors 

Grand total 

6.5% 

16.4% 

100.0%  100.0% 

100.0% 

100.0% 

100.0% 

Source:  for  LGF  and  EFG  -    EIF  quarterly  report  Q4/2016;  for  EYE:  data  from  EASME;  for  all  the  other 
action lines - datahub. Note: the datahub only contains information on calls for proposals; the sectors for 
the final beneficiaries of the calls for proposals have been attributed to LGF and EFG based on the NACE 
rev2  classification  and  for  all  other  action  lines  an  approximate  concordance  to  NACE  rev  2  was 
constructed based on the available sectoral information.  

Appendix C

 - List of documents analysed 

129 

 
 
C.1  
 SO1 – Access to Finance 
Table 43 List of documents analysed SO1 

List of documents analysed 

A Single Market Strategy for Europe - Analysis and Evidence, SWD(2015) 202 final  

Action plan to improve access to finance for SMEs – Communication from the Commission to the Council, European Parliament, Committee of the Regions, and to the European and Social Committee 
COM(2011) 870 final SEC (2011) 1527 final)  
CIP Results 2015 and 2016: CIP Results 180416.pdf (there are some updates in the web: http://ec.europa.eu/growth/access-to-finance/cip-financial-instruments_es) 

COM (2014) 4993 final, COSME Work Programmes (2014-16) 

COM (2015) 3767 final, COM (2016) 4344 final, COM (2014) 247 final, COM (2014) 8044 final, COM (2016) 63 final: Annexes to the Commission Implementing Decisions concerning the adoption of the 
work programmes for 2014 / 2015 / 2016 and the financing for the implementation of Programme for the Competitiveness of Enterprises and small and medium-sized enterprises. 
COM_Presentation_EFSI.pdf (https://era.gv.at/object/document/2505/attach/COM_Presentation_EFSI.pdf) 

DG GROW Presentation on draft internal regression analysis <<SMEloans_over1year.pptx>>  It gives an example on potential calculation of the gap in question. 

DIFFERENCES AND SIMILARITIES BETWEEN CIP AND COSME, Briefing Note IP/A/ITRE/2012-14 

Draft alternative conclusion provided by other colleagues. <<gap 375.docx>> It points at the same direction, i.e. almost bi-angulation, however, they might have used other data and/or methodology 

DRAFT REPORT on the implementation of the EFSI, European Parliament (http://www.europarl.europa.eu/sides/getDoc.do?type=COMPARL&reference=PE-
597.724&format=PDF&language=EN&secondRef=01) 
ECB is calculating external current financing gap, for example on page 4, table A and bottom of the  page: 
https://www.ecb.europa.eu/pub/pdf/other/accesstofinancesmallmediumsizedenterprises201611.en.pdf 
EFG calls for expression of interest, jointly with InnovFin: InnovFin Equity and COSME EFG Call for Expression of Interest at the bottom of the 
websitehttp://www.eif.org/what_we_do/equity/single_eu_equity_instrument/index.htm 
EFSI: an early proposal to extend and expand (2016); http://www.eca.europa.eu/Lists/News/NEWS1611_11/OP16_02_EN.pdf 

EIB Operations Evaluation - independent evaluation report on EFSI (2016); http://www.eib.org/attachments/ev/ev_evaluation_efsi_en.pdf 

EIF, 2015: The Economic Impact of EU Guarantees on Credit to SMEs. Evidence from CESEE Countries, http://www.eif.org/news_centre/publications/eif_wp_29_economic-impact-
guarantees_july15_fv.pdf 
Euro area bank lending survey https://www.ecb.europa.eu/stats/ecb_surveys/bank_lending_survey/html/index.en.html 

Europe's next leaders: the start-up and scale-up initiative. EC Communication COM(2016) 733 final 

European Small Business Finance Outlook (ESBFO), December 2016, EIF Research & Market Analysis, Working Paper 2016/37 
(http://www.eif.org/news_centre/publications/eif_wp_37_esbfo_dec16_final.pdf) 
Evaluation of EIF Own Resources Activity (2012); http://ec.europa.eu/dgs/economy_finance/evaluation/pdf/final_reprt_eval_eif.pdf 

Ex-ante assessment of the EU SME Initiative, SWD(2013) 517 final  

Final evaluation of the Competitiveness and Innovation Framework Programme 

130 

 
 
Final evaluation of the Entrepreneurship and Innovation Programme (April 2011) 

http://www.nefi.eu/index.php?eID=tx_nawsecuredl&u=0&g=0&t=1493922032&hash=aaab5903abcdbc636efefc7217c179b2fc0f8dca&file=fileadmin/Redaktion/Images/publications/Doc_08_-
_NEFI_HLM_Paris_-_Joint_Statement_of_the_NEFI_HLM_Final.pdf" 
https://ec.europa.eu/commission/publications/investment-plan-europe-evaluations-give-evidence-support-its-reinforcement_en; Page 9, second paragraph 

Impact Assessment accompanying the document Proposal for a regulation establishing a Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (2014 - 2020). 
COM(2011) 834 final, SEC(2011) 1452 final. 
Implementation Report 2012 EIPC of 14 March 2013 (Version revised 21 May 2013) 

InnovFin SMEG Implementation Status (http://www.eif.org/what_we_do/guarantees/single_eu_debt_instrument/innovfin-guarantee-facility/report-quarterly-innovfin-smeg-implementation-status.pdf) 

Interim evaluation of the Competitiveness and Innovation Framework Programme 

Interim evaluation of the Entrepreneurship and Innovation Programme (April 2009) 

Joint Committee Complementarity COSME LGF-InnovFin SMEG, Presentation 2015 

Key role of National Promotional Institutions in the EFSI SME Window, Joint Statement of the NEFI High Level Members (May 2016) 

Kraemer-Eis, H., Signore, S. and D. Prencipe (2016), The European venture-capital landscape: an EIF perspective, Volume I: The impact of EIF on the VC ecosystem, EIF Research and Market Analysis, 
Working Paper 2016/34. 
Latest annual report for GIF facility: CIP GIF Annual Report as at 30.09.2016.pdf 

Latest annual report for SMEG facility:  CIP SMEG 2016 Annual report.pdf 

LGF calls for expression of interest: bottom of the website http://www.eif.org/what_we_do/guarantees/single_eu_debt_instrument/cosme-loan-facility-growth/index.htm 

Regulation establishing a  Programme for the Competitiveness of Enterprises and small and medium-sized enterprises (COSME) (2014 - 2020). Regulation (EU) No 1287/2013 

Report on the public consultation under the Start-up Initiative 

Research for REGIO COMMITTEE - Maximisation of synergies between European Structural and Investment Funds and other EU instruments to attain Europe 2020 goals", 
http://www.europarl.europa.eu/RegData/etudes/STUD/2016/585872/IPOL_STU(2016)585872_EN.pdf 
Restoring EU Competitiveness, EIB (2016); http://www.eib.org/attachments/efs/restoring_eu_competitiveness_en.pdf 

Revision of the Regulation on the EFSI– towards an EFSI 2.0?, Briefing implementation appraisal, European Parliamentary Research Service (EPRS), September 2016; 
http://www.europarl.europa.eu/RegData/etudes/BRIE/2016/587342/EPRS_BRI%282016%29587342_EN.pdf 
SAFE survey 2016 

SBA Fact Sheet 2016 (http://ec.europa.eu/growth/smes/business-friendly-environment/performance-review-2016_en#interactive-sme-database) 

Scale-up manifesto. http://scaleupeuropemanifesto.eu/ 

Small Business Act for Europe: access to finance; https://ec.europa.eu/growth/access-to-finance_en 

Special report no 19/2016: Implementing the EU budget through financial instruments – lessons to be learnt from the 2007-2013 programme period; 
http://www.eca.europa.eu/Lists/ECADocuments/SR16_19/SR_FIN_INSTRUMENTS_EN.pdf 
Statistics from the access to finance website: web page of access to finance in European countries; http://europa.eu/youreurope/business/funding-grants/access-to-finance/index_en.htm; Latest data 
<<a2f stats.xlsx>> 

131 

 
Study on ''Got rejected? Real effects of not getting a loan'':https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp1960.en.pdf 

Study on 'Building Momentum in Venture Capital across Europe',  https://www.kfw.de/PDF/Download-Center/Konzernthemen/Research/PDF-Dokumente-Studien-und-Materialien/Building-Momentum-in-
Venture-Capital-across-Europe.pdf 
The audit of the SMEG Facility (2011); http://www.europarl.europa.eu/meetdocs/2009_2014/documents/cont/dv/sr4_/sr4_en.pdf 

The EFSI, maximising its potential,  European Political Strategy Centre (EPSC) Strategic Notes, Issue 11 (2016); https://ec.europa.eu/epsc/sites/epsc/files/strategic_note_issue_11.pdf 

The Shortage of Risk Capital for Europe’s High Growth Businesses (2017), https://www.afme.eu/globalassets/downloads/publications/afme-highgrowth-2017.pdf?dm_i=3TYX,674O,2T0PMT,LD2H,1 

Upgrading the Single Market: more opportunities for people and business, EC Communication, COM(2015), 550 final 

Web page Statistical Data Warehouse: Customizing the results of the Bank Lending Survey and the Survey on the Access to Finance of Enterprises (http://sdw.ecb.europa.eu/browse.do?node=9691105) 

Web Page: SME Performance Reviews 2015/2016 (https://ec.europa.eu/growth/smes/business-friendly-environment/performance-review-2016_en) 

Work Programme: They plan to audit 'EU’s venture capital interventions' and 'EFSI: management of investment plan and EU added value' as we can see from their priority tasks available. In addition 
they plan to produce an special report to assess the impact of the SMEG on access to finance for SMEs and the effectiveness of the SMEG in the 2007-2013 period. 
http://www.eca.europa.eu/Lists/ECADocuments/WP2017/WP2017_EN.pdf 

Source: Technopolis 

132 

 
 
 
 SO2 – Access to markets  

C.2  
Table 44 List of background documents SO2 

List of documents analysed 

BusinessEurope, (2011), “Are the Points of Single Contact truly making things easier for European companies” 

BusinessEurope, (2012), “BusinessEurope contribution on the redesign of the enterprise europe network (EEN)”  

Centre of Strategy & Evaluation Services, (2011), "Final Evaluation of the Competitiveness and Innovation 
Programme"  
Council of the European Union, (2015), "OUTCOME OF THE COUNCIL MEETING 3371 Council meeting 
Competitiveness (Internal Market, Industry, Research and Space)"  
DroneRules.EU, (2016), "Consortia response to: EASME Letter - Result of the Review of your COSME project 
680960 — DRONE-RULES.eu" 
EASME, (2009), "Promotion document: “An Introduction to the Enterprise Europe Network” 

EASME, (2013), "Enterprise Europe Network Activity report 2008-2012" 

EASME, (2013), "Enterprise Europe Network Progress Report 2013" 

EASME, (2014), "Brochure Enterprise Europe Network 2015 - SMEs step up a gear with the Enterprise Europe 
Network" 
EASME, (2014), "Template: EASME MULTI-PARTNER MODEL FRAMEWORK PARTNERSHIP AGREEMENT FOR THE 
ENTERPRISE EUROPE NETWORK (EEN FPA — MULTI)" 
EASME, (2015), "Your Europe Business Portal - Exit survey results 2015" 

EASME, (2016), "Enterprise Europe Network Progress Report 2015" 

EASME, (2016), "Grant Application - EU Japan Centre for Industrial Cooperation"  

EASME, (2016), "Letter - Result of the Review of your COSME project 680960 — DRONE-RULES.eu": 
Ares(2016)695609462913 
EASME, (2016), "PO Assessment Report/657846/LA IPR SME HD": Ares(2016)7015710 

EASME, (2016), "PO Assessment Report/657914/SEA HD 2015-2017": Ares(2016)6286280 

EASME, (2016), "Your Europe Business Portal - Exit survey results 2016" 

EASME, (2017), "Coaching under the SME-Instrument": https://ec.europa.eu/easme/en/coaching-under-sme-
instrument. 
EASME, (2017), "Progress report - DRONE-RULES.eu - RPAS rules regulation portal": Ares(2017)327875 

EASME, (2017), "Progress report - Request for revision of deliverable submission - DRONE-RULES.eu - RPAS 
rules regulation portal": Ares(2017)740655 
Ecorys and IDF GHK, (2014), "SMEs' access to public procurement markets and aggregation of demand in the 
EU"  
EU Japan Centre, (2012), "Vulcanus in Japan - Key figures" 

EU Japan Centre, (2014), "EU Japan Centre - Activity Report 2014" 

EU Japan Centre, (2014), "Presentation: EU Japan Centre - Activity Report 2014" 

EU Japan Centre, (2015), "EU Japan Centre - Activity Report 2015" 

EU Japan Centre, (2015), "Presentation: EU Japan Centre - Activity Report 2015" 

EU Japan Centre, (2016), "EU Japan Centre - Work Programme 2016" 

EU Japan Centre, (2016), "Traineeship in Japan for students from the EU & COSME* countries" 

EU Japan Centre, (2017), "Application form: Vulcanus Programme" 

EU Japan Centre, (2017), "Vulcanus in Japan - Frequent Questions and Answers" 

EUROCHAMBRES (2011) “Enterprise Europe Network: Chambers’ views on its services post 2014”  

EUROCHAMBRES, (2012), “Response to the Commission consultation on the Enterprise Europe Network”  

Euroepan Commission, (2015) "Call for Expressions of Interest ‘Business Cooperation Centres’ in third countries 
for the Enterprise Europe Network 2015-2020 - Annex 1 EU Business Support in Markets Outside the EU"  
Euroepan Commission, (2015) "Call for Expressions of Interest ‘Business Cooperation Centres’ in third countries 
for the Enterprise Europe Network 2015-2020": COS-Art-7-001 
Eurofound (2015), “Job creation in SMEs ERM annual report 2015” 

European Commission (2001), “European Charter for Small Enterprises – Annual Implementation Report” 

European Commission, (2005), “Proposal for a Decision of the European Parliament and of the Council 
establishing a Competitiveness and Innovation Framework Programme (2007-2013)": COM(2005) 121 final 

133 

 
 
 
European Commission, (2006), "Call for proposals: Services in support of business and innovation": 2006/C 
306/07 
European Commission, (2006), "DECISION No 1639/2006/EC OF THE EUROPEAN PARLIAMENT AND OF THE 
COUNCIL of 24 October 2006 establishing a Competitiveness and Innovation Framework Programme (2007 to 
2013)": 2006/C 306/07 
European Commission, (2007), "Call for proposals: Services in support of business and innovation": 2007/C 
227/04 
European Commission, (2007), "Submission Set - Call for proposals (ENT/CIP/070001a): Services in support of 
business and innovation" 
European Commission, (2008), "Points of Single Contact: Doing business made easier": 
http://ec.europa.eu/internal_market/eu-go/docs/psc_en.pdf 
European Commission, (2009), "Interim evaluation of the Enterprise Europe Network" 

European Commission, (2010)," Evaluation of EIP indicators" 

European Commission, (2011), "Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE 
COUNCIL establishing a Programme for the Competitiveness of Enterprises and small and medium-sized 
enterprises (2014 - 2020)": COM(2011) 834 final, SEC(2011) 1452, SEC(2011) 1453 
European Commission, (2012), "Competitiveness and Innovation Framework Programme Performance from 2007 
to 2012" 
European Commission, (2012), "Implementation of the Small Business Act for Europe (SBA) and 
Entrepreneurship Policies at Local and Regional Level" 
European Commission, (2013), "CIP Performance Report": JCM-02-2013 

European Commission, (2013), "REGULATION (EU) No 1287/2013 OF THE EUROPEAN PARLIAMENT AND OF THE 
COUNCIL of 11 December 2013 establishing a Programme for the Competitiveness of Enterprises and small and 
medium-sized enterprises (COSME) (2014 - 2020) and repealing Decision No 1639/2006/EC" 
European Commission, (2014), "COSME Work Programme 2014 - Commission Implementing Decision concerning 
the adoption of the work 
programme for 2014 and the financing for the implementation of Programme for the 
Competitiveness of Enterprises and small and medium-sized enterprises" 

European Commission, (2014), "Statistical Report About Your Europe Business Portal 2014" 

European Commission, (2014), “Call for proposals – COSME Enterprise Europe Network - COS-EEN-2014-2-01” 

European Commission, (2014), “Call for proposals – COSME Enterprise Europe Network - COS-EEN-2014-2-04” 

European Commission, (2014), “Call for Proposals Facilitating access to regulation for light Remotely Piloted 
Aircraft Systems (RPAS)": COS-RPAS-2014-2-03 
European Commission, (2014), “Call for proposals for China, Latin America And South-East Asia IPR SME 
Helpdesks": COS-WP2014-2-05 
European Commission, (2015), "COSME Work Programme 2015 - Commission Implementing Decision concerning 
the adoption of the work 
programme for 2015 and the financing for the implementation of Programme for the 
Competitiveness of Enterprises and small and medium-sized enterprises" 

European Commission, (2015), "Enterprise Europe Network survey 2016 – Your Europe Business Portal 2015" 

European Commission, (2015), "PERIODIC REPORT COSME PROGRAMME - Action Title: China IPR SME Helpdesk" 

European Commission, (2015), "PERIODIC REPORT COSME PROGRAMME - Action Title: Latin America IPR SME 
Helpdesk" 
European Commission, (2015), "PERIODIC REPORT COSME PROGRAMME - Action Title: South-East Asia IPR SME 
Helpdesk" 
European Commission, (2015), "Proposal Evaluation Summary Report - DRONE-RULES.eu - RPAS rules 
regulation portal - COS-RPAS-2014-2-03": Ares(2015)2940919 
European Commission, (2015), "Statistical Report About Your Europe Business Portal 2015" 

European Commission, (2015), "Study on good practices in the area of compliance assistance and compliance 
schemes FRAMEWORK CONTRACT-ENTR/300/PP2013/FC" Ares (2015)3332549 
European Commission, (2015), "The Performance of the Points of Single Contact An Assessment against the PSC 
Charter" 
European Commission, (2015), “Flash Eurobarometer 421 – Internationalisation of Small and Medium-sized 
Enterprises” 
European Commission, (2015), “GRANT AGREEMENT NUMBER — 680960 — DRONE-RULES.eu": 
Ares(2015)3949307 
European Commission, (2015), “Innobarometer 2015 – The innovation trends at EU enterprises” 

European Commission, (2016), "AMENDMENT Reference No AMD-657846-4 Grant Agreement: 657846 — Latin 
America IPR SME Helpdesks (LA IPR SME HD)": Ares(2016)3256986 
European Commission, (2016), "CALL FOR PROPOSALS - Supporting the businesses' expansion in the Internal 
Market by improving Points of Single Contact – Increasing awareness among entrepreneurs about the Points of 
Single Contact": Ares(2016)4745300 
European Commission, (2016), "COSME Work Programme 2016 - Commission Implementing Decision concerning 
the adoption of the work 
programme for 2016 and the financing for the implementation of Programme for the 
Competitiveness of Enterprises and small and medium-sized enterprises" 

European Commission, (2016), "Description of the action: EU Japan Centre for Industrial Cooperation - 
GRO/SME/16/B/041": Ares(2016)2787248 
European Commission, (2016), "Enterprise Europe Network survey 2016 – Your Europe Business Portal 2016" 

134 

 
European Commission, (2016), "Europe’s Next Leaders: The Start-Up And Scale-Up Initiative": COM(2016) 733 
final 
European Commission, (2016), "GRANT AGREEMENT NUMBER — 657914 — SEA HD 2015-2017": 
Ares(2015)1197896 
European Commission, (2016), "Statistical Report About Your Europe Business Portal 2016" 

European Commission, (2016), "Technical Final Report - Study on Good Practices in the Area of Compliance 
Assistance and Compliance Schemes"  
European Commission, (2016), “Call for proposals for Improved Points of Single Contact": COS-SPOC-2016-02-
03 
European Commission, (2016), “Call for proposals for Improving access to SMEs in public procurement": COS-
APP-2016-2-05 
European Commission, (2017), "Enterprise Europe Network - Key results & figures for COSME committee" 

European Commission, (2017), "Explanatory Memorandum of the EC Proposal for a Regulation of the European 
Parliament and of the Council on establishing a single digital gateway to provide information, procedures, 
assistance, and problem solving services": COM(2017) 256 final 
European Commission, (2017), "Official website on the EU Single Market Scoreboard": 
http://ec.europa.eu/internal_market/scoreboard/performance_by_governance_tool/points_of_single_contact/ind
ex_en.htm. 
European Commission, (2017), "Presse release: Aviation: Commission is taking the European drone sector to 
new heights" 
European Commission, (2017), "Single Market Scoreboard - Points of Single Contact": 
http://ec.europa.eu/internal_market/scoreboard/_archives/2013/07/performance_by_governance_tool/points_of
_single_contact/index_en.htm 
European Commission, (2017), "SME Performance Review and Small business act Factsheets": 
https://ec.europa.eu/growth/smes/business-friendly-environment/performance-review-2016_en 
European Commission, (2017),"Proposal for a Regulation of the European Parliament and of the Council on 
establishing a single digital gateway to provide information, procedures, assistance, and problem solving 
services": COM(2017) 256 final 
OECD (2004), “Promoting Entrepreneurship and Innovative SMEs in a Global Economy - Towards a More 
Responsibleand Inclusive Globalisation – 2nd OECD Ministerial Conference on SMEs – June 2004 – Istanbul 
Turkey” 
SESAR Joint Undertaking, (2016), "European Drones Outlook Study - Unlocking the value for Europe" 

t33, SWECO and OIR, (2012), "Implementation for the Small Business Act for Europe (SBA) and 
Entrepreneurship Policies at Local and Regional Level" 
Technopolis Group (2015), "Final evaluation of the impact of the Enterprise Europe Network 2008-2014" 

Technopolis Group, (2009), "Interim Evaluation of the Competitiveness and Innovation Framework Programme 
(2007-2013)" 
Technopolis Group, (2015), "Evaluation of the three years of operation (2011 – 2013) of the Executive Agency 
for Competitiveness and Innovation (EACI)" 
The British Standards Institution (BSI) and WIFO, (2015), "Proposal for an analytical study - Study on Good 
Practices in the Area of Compliance Assistance and Compliance Schemes"  
The British Standards Institution (BSI) and WIFO, (2016), "Final Report - Study on Good Practices in the Area of 
Compliance Assistance and Compliance Schemes"  
The British Standards Institution (BSI) and WIFO, (2016), "Mid-Term Report - Study on Good Practices in the 
Area of Compliance Assistance and Compliance Schemes"  
UEAPME, (2012), “Position Paper - proposals for a renewed EEN”  

 SO3 – Favourable environment 

C.3  
Table 45 List of background documents SO3 

Thematic 
area 
Cluster 
internationali
sation  

List of main background documents used  

Analysis of the consultation launched by ECCP amongst the ESCP-4i Partnerships in June 2016 
(published Oct 2016) 
Background documents: 

Call for Proposals – Clusters Go International (COS-CLUSTER-2014-3-03) 

European Cluster Collaboration Platform (clustercollaboration.eu) resources:   

Evaluation of European Cluster Initiatives managed by DG Enterprise and Industry (2015), 
performed by Technopolis Group 
Iszak, K. (April 2016): European Strategic Cluster Partnerships Going International: common 
opportunities for internationalisation, Technopolis Group paper.    
Monitoring / analysis documents published on the European Cluster Collaboration website, 
containing statistics on the ESCP4i clusters  
Monitoring reports of the implementation of the ESCP-4i projects, provided by EASME 

SME Policy 

2014 Report on public consultation on SBA  

2016 SME Envoys report on the implementation of SME Test in EU countries and presentations of 
the participants in the technical seminar on the SME Test (Oct 2016) 
Commission Decision of 4.7.2012, setting up the group of experts on Enterprise and SME 
Policies, C(2012)4386final  

135 

 
Documents of the SME Envoys Network, 2014-2016 

European Parliament reports on the European Semester for economic coordination 2016, 2015 
and 2014   
Further reports and sources   

Meeting minutes of the SME Envoys Network available on the European Commission 
Transparency Register,  
http://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupDetail&groupID=26
66  
Policy documents  

Results of the Annual Growth Survey 2014-2017 

SME Envoys mission description  

SME Performance Review resources: Small Business Act Factsheets, Annual Report on European 
SMEs 2014-2016 
SME Policy website of DG GROW 

The SBA for Europe and SBA Review, 2011 

The Single Market Strategy for Europe  

Tourism  

Tourism actions documents  

Accessible Tourism Services report (2015)  

Ad-hoc grant (2014-2016) awarded to the European Travel Commission for Destination Europe 
2020 Campaign  
Call for Proposals to select EDEN destinations COS-WP2014-03-15.01 and call COS-TEDEN-2014-
3-15 for communication support   
Calls for proposals for the tourism sector grants actions: COS-WP2014-3-15.02 (synergies with 
high-end industries and creative industries), COS-WP2014-3-15.03 (diversifying EU tourism offer 
through sustainable products), COS-TOUR-2015-3-04 (supporting competitive and sustainable 
growth in the tourism sector – 3 themes);   
CIP Evaluation (2013), including references to actions on Tourism  

COS-TFLOWS-2014-3-15 

COS-TOUR-2015-3-04-1 

COS-TSUST-2014-3-15 

COS-TSYNER-2014-3-15 

DG GROW – Guide to EU funding for the Tourism sector 

DG GROW, Tourism website   

Eurobarometer surveys of the Tourism sector  

European Commission, 2010 Communication on Tourism   

European Travel Commission reports and statistics  

Mapping of skills and training needs for accessibility in tourism services (2014)  

Monitoring reports of the calls (a sample of 4 reports per call, due to the lack of a synthesised 
report on the progress of the actions).   
OECD reports on Tourism sector (e.g. 2016 OECD report on Tourism Trends and policies)   

Policy studies, reports and websites:    

Travel patterns of accessible tourism in Europe (2013) 

Virtual Tourism Observatory 

World Tourism Organisation UNWTO – World Tourism Barometer   

e-skills  

Cattaneo, G., Aguzzi, S, et. Al, (2014): Digital Agenda for Europe – Engagement for Growth and 
Jobs  
Digital Agenda for Europe (2010). 

E-leadership skills: Second draft interim report, 2016, together with the Technical Assessment 
Sheet e-leadership; Final report (May 2017)    
e-Skills for Jobs Campaign website, http://eskills4jobs.ec.europa.eu/  

E-skills for Jobs Campaign, Final Report (Part 1 and 2), 2016   

e-skills for Jobs Manifestos (2014-2016)  

Empirica (2015): e-Skills in Europe: Trends and Forecasts for the European ICT Professional and 
Digital Leadership Labour Markets 

136 

 
Empirica et al, (2015): Final Report - e-Leadership Skills for Small and Medium Sized Enterprises 

European Framework for the IT profession: Interim report, June 2016 together with the  
Technical Assessment Sheet IT profession; Final report & brochure (forthcoming, May 2017)  
German and Austrian SME Envoys report (2016): Avoiding skills shortages and mismatches – 
New strategies for meeting companies’ skilled labour needs  
Innovation Value Institute, IDC, Empirica, CEPIS (2014): e-Skills: The International dimension 
and the impact of globalisation  
Policy Documents  

Project implementation documents received from DG GROW 

Reports  

Reports of the Grand Coalition for Digital Jobs     

S. Robinson, L. Hendrix, B. Hanny, W. B. Korte, T. Hüsing (2015): New Curricula for e-
Leadership Skills. Guidelines and Quality Lables for new Curricula for e-Leadership Skills in 
Europe. Final Report. Bonn. http://eskills-lead.eu/documents.html  
T. Hüsing, W.B. Korte, E. Dashja (2015): E-skills and e-leadership skills 2020. Trends and 
forecasts for the European ICT professional and digital leadership labour market. Empirica 
Working Paper. Bonn. 
The Commission Communication of 2007, 'e-Skills for the 21st Century: Fostering 
Competitiveness, Growth and Jobs' - a long term e-skills agenda for the period 2008-2010. 
The Employment Package (2012)  

The European e-Competence Framework  -  a reference of 40 skills required in the ICT workplace 
(http://www.ecompetences.eu/).  
Websites and further resources:  

Sectoral 
competitiven
ess (except 
for tourism) 

www.leadership2015.eu and  www.eskills-lead.eu - website of European Conferece on digital and 
key enabling technologies skills 2015 and of the  service contract to promote e-leadership in 
Europe (LEAD) 
Call for Tenders specifications on Establishment of the "European Sustainable Chemicals Support 
Service", EASME/COSME/2015/01,7, Phase 1 and Phase 2 
Presentation of results of the Call for Proposals on design-based consumer goods, received from 
DG GROW  
The Construction 2020 Action Plan and reports on its implementation from DG GROW website.  

KETs 

Projects’ implementation documents, websites  and policy documents   

 EC Communication 2016, on “Digitising European Industry. Reaping the full benefits of a Digital 
Single Market”  
Coordination framework for Digital Single Market  

DG GROW, KETs Tools Website     

Digital Agenda for Europe (2010). 

KETs High Level Work Group reports – final results and recommendations  

KETs strategy, 2015   

The Commission Communication of 2014, 'An Industrial Renaissance for EU”     

C.4  
 SO4 – Entrepreneurship 
Table 46 List of documents analysed SO4 

List of documents analysed 

Annex 1 to the Commission Implementing Decision concerning the adoption of the work programme for 2014 and 
the financing for the implementation of Programme for the competitiveness of Enterprises and small and 
medium-sized enterprises.  C(2014) 247 final. 
Annex 1 to the Commission Implementing Decision concerning the adoption of the work programme for 2015 and 
the financing for the implementation of Programme for the competitiveness of Enterprises and small and 
medium-sized enterprises.  C(2014) 8044 final. 
Annex 1 to the Commission Implementing Decision concerning the adoption of the work programme for 2016 and 
the financing for the implementation of Programme for the competitiveness of Enterprises and small and 
medium-sized enterprises.  C(2016) 63 final. 
Annex 1 to the Commission Implementing Decision concerning the adoption of the work programme for 2017 and 
the financing for the implementation of Programme for the competitiveness of Enterprises and small and 
medium-sized enterprises.  C(2016) 7033 final. 
Attitudes towards the planned EU pilot project - Erasmus for Young Entrepreneurs; Report 3: Survey among 
young would-be and existing entrepreneurs in all EU Member States: Analytical Report” produced by the Gallup 
Organisation for Directorate-General for Enterprise and Industry, November 2007 
Commission staff working document accompanying the document Report from the Commission to the European 
Parliament and the Council.  COSME – Programme for the Competitiveness of Enterprises and SMEs 2014-20 – 
Monitoring Report 2014.  SWD (2016) 274 final. 
Communication “Think Small First – A ‘Small Business Act for Europe’ COM(2008) 394 final 

137 

 
COSME Monitoring Report 2015. 

COSME programme documentation (annual work programmes, texts of the calls for proposals and tenders, 
annual monitoring reports).  Key text referenced included: 
COSME.  Programme for the Competitiveness of Enterprises and SMEs 2014-2020. Monitoring Report 2014. 
Annex 1 – Details of COSME Actions 2014.  European Commission. 
EIP Implementation Report 2011.  EIPC of 15 March 2012.  (European Commission, 2012) 

EU and other international policy studies, evaluations and assessments on the topics.  Key text referenced 
included: 
EU policy documents, strategies and action plans.  Key text referenced included: 

European Commission, Entrepreneurship 2020 Action Plan 

European Commission: Call for Proposals, Migrants Entrepreneurship Support Schemes, Europe: 2016 

Eurostat, 'Statistics in focus: the entrepreneurial gap between men and women' (30/2007) 

Evaluation and Analysis of Good Practices in Promoting and Supporting Migrant Entrepreneurship. EU conference 
on migrant entrepreneurship.  Background Paper. 23 February 2016.  European Commission. 
Feasibility study for the Pilot Action “Erasmus for young entrepreneurs" Part A: Scenarios and Costs. A Final 
Report to DG Enterprise of the European Commission (ECOTEC Research and Consulting, 2008) 
Final evaluation of the Entrepreneurship and Innovation Programme – Final report. (CSES, EIM, April 2011) 

Final Report. The future of Europe’s rural periphery, the role of Entrepreneurship in responding to employment 
problems and social marginalization.  EU research on social science and humanities.  European Commission (EUR 
21704) 
Green Paper – Entrepreneurship in Europe. COM(2003) 27 final 

Interim evaluation of the Erasmus for Young Entrepreneurs Pilot project / Preparatory action (CSES, 2011) 

Past studies and evaluations (of CIP, EIP and EYE).  Key text referenced included: 

Project level information – including information accessed from the project websites 

Report from the Commission to the European Parliament and the Council.  COSME. Programme for the 
Competitiveness of Enterprises and SMEs 2014-2020. Monitoring Report 2014. COM(2016) 526 final. 
Statistical Analysis of the Erasmus for Young Entrepreneurs (EYE) Programme (Technopolis, 2014) 

138 

 
 
 
Appendix D

 – Lists of interviewees 

 SO1 – Access to Finance 

D.1  
Table 47 Interviews performed SO1 

Interview 
category 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 

A: 
Representatives 
from EU 
institutions 

A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
A: 
Representatives 
from EU 
institutions 
B: Intermediary 
organisations 

Interview 
ID 
A-SO1-1 

Last Name 

Schreiber 

First 
Name 
Kristin 

Institution 

Role 

DG GROW 

Director, H, COSME program 

A-SO1-2 

Cristea 

Ciprian 

DG GROW 

Head of Unit, H3, Financial 
Instruments 

A-SO1-3 

Diss 

Martine 

DG GROW 

Deputy head of Unit, H3, Financial 
Instruments 

A-SO1-4 

Bartels 

Astrid 

DG GROW 

Team leader 

A-SO1-5 

Squilloni 

Matteo  

EIF 

Mandate Guarantees 

A-SO1-6 

Boregas 
Amaro 

Luis 

EIF 

Mandate VC investments 

A-SO1-7 

Gresko 

Patrick 

EIF 

Investments, Technology and 
innovation 

A-SO1-8 

Manca 

Silvia 

EIF 

A-SO1-9 

Mai 

Gunnar 

EIF 

Mandate Management, 
Institutional Business 
Development 

Head of Division, EU Guarantee 
Facilities 

A-SO1-10 

Gonzalez 
Martin 

David 

EIF 

Institutional Business 
Development Manager 

A-SO1-11 

Appel 

Stefan 

DG REGIO 

A-SO1-13 

Clausen 

Merete 

DG ECFIN 

A-SO1-14  Muent 

Gunnar 

EIB 

A-SO1-15 

Cannenterre  Karen 

EIB 

A-SO1-16 

Stoykov 

Hristo 

EIB 

Head of Unit B3 - Financial 
Instruments and relations with 
International Financial 
Institutions within Directorate 
'Policy' 
Head of Unit 'Financing of 
innovation, competitiveness and 
employment policies' within 
Directorate 'Treasury and financial 
operations' 
Director of innovation and 
Competitiveness 

Intermediated Financing to Micro, 
Small and Medium Size 
Entreprises 

Equity, New Products and Special 
Transactions 

B-SO1-1 

Klein 

Germaine  European 

EAPB interlocutor 

Association of 

139 

 
 
 
B: Intermediary 
organisations 

B-SO1-2 

Sturm 

Katrin 

B: Intermediary 
organisations 

B-SO1-3 

Tschofen 

Nina 

B: Intermediary 
organisations 

B: Intermediary 
organisations 

B-SO1-4 

Collins 

Michael  

B-SO1-5 

Huemer 

Gerhard 

AECM Secretary-General  

Coordinator 

Chief Executive 

Research and Innovation 

Public Banks 
(EAPB) 

European 
Association of 
Guarantee 
Institutions 
(AECM) 
Network of 
European 
Financial 
Institutions for 
Small and 
Medium Sized 
Enterprises 
(NEFI) 
InvestEurope 

European 
Association of 
Craft, Small 
and Medium-
sized 
Enterprises 
(UEAPME) 

B: Intermediary 
organisations 

B-SO1-6 

Cloquet 

Daniel 

BusinessEurope  Director SMEs and 

entrepreneurship 

 SO2 – Access to market 

D.2  
Table 48 Interviews performed SO2 

Stakeholder 
type 
EU officials 

Organisation 

EC – DG GROW 

EASME 

EC – DG RTD 

EC – DG DEVCO 
EC – DG GROW 

EC – DG REGIO 
EC – Service for Foreign Policy Instruments (FPI) 

Center for Technology Transfer and Innovation 
(CTT)  
Hanze Hogeschool 
Croatian Chamber of Economy 
ARC Consulting 
Agencia IDEA 
IAPMEI 
CCI Auvergne Rhone Alpes 
Eurosportello-Servizi Informativi Per Le Imprese 
Fundatia Centrul Roman Pentru Interprinderi Mici Si 
Mijlocii 
Hessen Trade & Invest Gmbh 
Investment and Development Agency of Latvia 
Politechnika Wroclawska 
Scottish Enterprise 
Technologicke Centrum Akademie Ved Ceske 
Republiky 
Tillvaxtverket 
Fonden Vaeksthus Hovedstadsregionen 
Wirschaftskammer Oesterreich 
Ss Cyril and Methodius University in Skopje 
Camera de Comert Si Industrie A Republicii 
Moldova 
UEAPME 

BusinessEurope 

140 

Beneficiaries 
(intermediaries) 

Umbrella 
organisation 

Name 

Giacomo MATTINO (Head of Unit – H2) 
Patrick DE SMEDT (Deputy Head of Unit – 
H2) 
Jose PUIGPELAT (Deputy Head of Unit – 
A1) 
Muriel DE GRANDE (Team leader 
evaluation and monitoring) 
Steve ROGERS (Deputy Head of Unit – 
B3)  
Soyes THIERRY (Cooperation Officer) 
Christophe GUICHARD (Deputy Head of 
Unit – F2) 
Katje REPPEL (Deputy Head of Unit – G1) 
Daniel VAN ASSCHE (Programme Officer 
– EU Policies) 
Spela STRES 

Matthijs PLIJNAAR 
Vesna TORBARINA 
Daniela TCHONKOVA 
José A. Pascual SANCHEZ 
Vanda NARCISO 
Salvatrice BUFALINO 
Maria Christina RAFFONE 
Cristian ORMINDEAN 

Khaled SNOUBER 
Jekaterina VANAGA 
Agniezska TURYNSKA-GMUR 
Caroline GRAY-STEPHENS 
Petr HLADIK 

Tony MEURKE 
Simon POULSEN 
Cristoph HUTER 
Beti KOSADINOVSKA 
Ludmila BOTOSAN 

Luc HENDRICKX 
Veronique WILLEMS 
Daniel CLOQUET 

 
EBN (Innovaiton Network) 
EURADA 

Giodrano DICHTER  
Esteban PELAYO VILLAREJO 

Eurochambers 

Silvia Caneva 

 SO3 – Favourable environment 

D.3  
Table 49 Interviews performed SO3 

Stakehol
der type 

EU 
officials 

Organisation 

Name 

EC – DG GROW 

Maarit Nyman (Deputy Head of Unit – H1) 
Christophe Guichard (Unit – F2)    
Andre Richier (Unit – F3 – dealing with E-skills) 

Heidi Moens (Unit – F3 – dealing with KETs) 

Domenico Lombardi  

Iuliana Aluas (Head of Unit – F4) 
Piedad Rivas, F4 (together with Iuliana Aluas)  

Timoteo de la Fuente (Unit – D2)  

EASME 

EC – DG RTD 
EC – DG REGIO 

DG CONNECT 

Valentino Izzo, Giovanna D’Addamio, Antonis Christopoulos  
Ramunas Kuncaitis (Head of Sector A.1.4.- 
Entrepreneurship  / for Clusters)   
Ana Grigore,  Unit D1  (KETs) 
Katja REPPEL (Deputy Head of Unit – G1) – together with 
SO2 
Joanna Mouliou (D1)  
Helen Kopman, Deputy Head of Unit, Startup Europe   

Topics  

SME Policy  
Clusters 
E-skills  

KETs 

SME Policy 

Tourism 
Design-based 
consumer goods  

Sectoral 
competitiveness – 
chemical regions 
service 
Tourism   
Clusters  

KETs  
Clusters, KETs  

Tourism  
SME Policy 

DG EAC 

Barbara Gessler, Head of Unit, Creative Europe Programme 

Tourism 

DG TRADE 

Martin Pilsner  

Beneficiari
es 
(intermedi
aries) 

European Travel 
Commission  
European 
Strategic Cluster 
Partnerships  
coordinators 

Eduardo Santander, executive director, and Teodora 
Marinska 
Vladimir Gumilar, ECCA 
Emilie Romeo, BioX 
Patrick Vuillermoz, WIINTECH 
Anna Naydenova, ICT Cluster Bulgaria 

National 
and 
regional 
governme
nts 

Experts 

Umbrella 
organisati
on 

National 
representatives 
Tourism 

National 
Representative 
SME Policy  
Regional 
Representative 
UEAPME 

Johan Lecocq, Silicon Europe 

Lithuania: Lidjia Bajaruniene 

Romania: Iulia Dangulea 

Portugal: Joanna Pinto Coelho 
Gabriele Schmid, Federal Ministry of Science, Research and 
Economy, Austria 

Jan Larosse, Expert, DG Regio and Flemish Government 

Luc HENDRICKX, responsible for SBA & SME policy– 
together with SO2 

Business Europe  Daniel Cloquet, Director, SME competitiveness 

Mark Pattinson, INNO AG 

SME Policy, Clusters, 
Sectoral 
Competitiveness 

Tourism  

Clusters  

Tourism  

Tourism 

Tourism 
SME Policy  

Clusters and Sectoral 
Competitiveness  
SME Policy 

SME Policy and 
Sectoral 
Competitiveness 
Clusters  

European 
Cluster 
Collaboration 
Platform  
TCI Network 
members  
SPIRE  

CIONET 
Netherlands 

Joep Browers,  Brainport Development 

Clusters  

Angels Orduna  

Frits Bussemaker, ICT Programme Manager  

Sectoral 
competitiveness  
E-skills  

141 

 
 
ECDL 
Foundation 
Academy Cube  

CEPIS 

Austeja Trinkunaite 

Lisa Keiler  

Fiona Fanning 

E-skills  

E-skills  

E-skills 

Table 50  Interviews not performed SO3 

Type 

Org. 

Name 

EU Officials 

EC- DG 
CONNECT  

EC – 
INTERREG  

Alexander Riedl   

Ronan Burgess  

KETs  

Magdalini Anagnostou 

Clusters  

Topics  

E-skills  

Status  

Refused   

Did not reply despite 
repeated contacting 
Did not reply, replaced with 
DG REGIO – Joanna Mouliou 
Niessler (D1)  

No answer despite emails 
and phone calls  

Refused, postponed to after 
15 June and refused again   
Postponed a couple of times 
then no answer 
No availability before end of 
June. Accepted for 20 June, 
when phone call and emails 
weren’t answered  
Refused. Team attempted to 
replace with Mateja 
Dermastia, Slovenian cluster 
expert, who was not 
available 
No answer despite repeated 
emails and phone calls.   

No answer despite repeated 
emails and phone calls     
Replaced with Fiona Fanning, 
CEPIS   

National 
governments 

Umbrella 
Organisations  

National 
representatives 
Tourism 
NECSTOUR  

Poland : Ewelina 
Macierzynska 

Tourism 

Cristina Nunez 

Tourism  

Eurochambers 

Slivia Caneva 

SME Policy 

TCI Network 
members (x3) 

Patricia Valdenebro, 
CEO, TCI Network   

Clusters  

Dorota Fraczek, Polish 
Agency for Enterprise 
Developent,  

Clusters  

EFFRA 

Zeljko Pazin 

KETs  

CECIMO 

Filip Geerts 

KETs 

EIT Digital 

Anders Flodstrom, 
director education 

E-skills  

Source: Technopolis 

 SO4 – Entrepreneurship 

D.4  
Table 51 Interviews performed SO4 

Group 
Programme 
management 

Intermediary 
organisations 

Beneficiaries – 
Host companies 

Name 
Michael Berz 
Patrick Klein 
Simone Baldassari 

Katerina Nedjlova 
Ivelina Fedulova 
Pascale Gaucher 
Marie Fallgaard 
Nadine Bettens 

Yavor Profirov  

Blanka  Bendlová 
Pamela Fombuena  

Thomas Berger  
Knieža Aivaras  

Susanne  Marklund 
Anna  Pajarón 

Alexandra  Yaghil 
Daniele Trevisan 
Guerric Gautier 
Francisco Sarrias 
Oliver Rauch 

Organisation 
DG GROW F3 
DG GROW F2 
DG GROW F2 

DG GROW H2 
Eurochambres 
DG GROW 
Aalborg University 
Agence Bruxelloise pour 
l’Entreprise  
Bulgarian Management 
Association 
JCI Czech Republic, z.s. 
Chambre de commerce et 
d'industrie de région Paris Ile-
de-France 
Hochschule Fulda 
Kaunas Chamber of Commerce, 
Industry and Crafts 
 European Minds 
Cambra Oficial de Comerç, 
Indústria I Serveis de Terrassa 
Southbank Technopark 
Kairos Europe 
Ozfair 
MSI 
Cow Style 

142 

Relevance 
Digital entrepreneurship  
Social entrepreneurship 
Various entrepreneurship 
actions 
EYE 
EYE Support office 
Women’s entrepreneurship 
EYE IO - Denmark 
EYE IO - Belgium 

EYE IO - Bulgaria  

EYE IO - Czech Republic 
EYE IO - France 

EYE IO - Germany 
EYE IO - Lithuania 

EYE IO - Sweden 
EYE IO - Spain 

EYE IO - UK 
EYE HE - Belgium 
EYE HE - Spain 
EYE HE - Austria 
EYE HE - UK 

 
 
 
Beneficiaries – 
New 
Entrepreneurs 

Other 
stakeholders 

Martin Stevens 
Matteo  Schipsi 
Alexandra  Nicorici  
Alberto Rodriguez Arias 
Heidi Storr 
Vassil Tzanov 
Matteo  Schipsi 
Efka Heder 

Maria Lustri 
Xavier Cobbaers 

IT IS 3D 
n/a 
n/a 
n/a 
n/a 
n/a 
n/a 
South East European Centre for 
Entrepreneurial Learning 
WES 

Olena Beck 

European Training Foundation 

EYE NE - Italy 
EYE NE - Romania  
EYE NE - Spain 
EYE NE - Germany 
EYE NE - Bulgaria  
EYE HE - Belgium 
EYE HE – Spain 
Advisory committee 

Advisory committee 
Contractor – women’s 
entrepreneurship 
Advisory committee 

[

C

a

t

a

l

o

g

u

e

n

u

m

b

e

r

]

143 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
             
 
                                
 
 
Annex 4: Interim Evaluation of the International Intellectual Property 
Rights (IPR) SME Helpdesk, a specific COSME measure 

Table of contents 

1. 

2. 

3. 

INTRODUCTION ............................................................................................................................ 147 

BACKGROUND TO THE IHD MEASURE ................................................................................... 147 

STATE OF PLAY ............................................................................................................................. 151 

4.  METHODOLOGY ............................................................................................................................ 155 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ........................................ 158 

6. 

CONCLUSION ................................................................................................................................. 196 

7.  ANNEXES ........................................................................................................................................ 198 

144 

 
 
 
 
Glossary 

Term or acronym 

Meaning or definition 

AHK 

ASEAN 

ASEANHD 

BCCs 

CIBEPYME 

CIP 

CNHD 

COSME 

EABC 

EASME 

EC 

ECCP 

EEN 

EFTA 

German Chambers of Commerce Worldwide Network 

Association of Southeast Asian Nations  

ASEAN helpdesk  

Business Cooperation Centres 

Plataforma Iberoamericana de Propiedad Industrial Dirigida a 
Empresas 

Competitiveness and Innovation Programme 

China IPR SME Helpdesk 

Programme for the Competitiveness of the Enterprises and Small 
and Medium-sized Enterprises 

EU Business Centres in Thailand  

The Executive Agency for Small and Medium-sized Enterprises 

European Commission 

European Cluster Collaboration Platform 

Enterprise Europe Network 

European Free Trade Association 

EKONID 

EU Business centres in India 

ESCPs 

EUIPO 

EVBN 

FAQ 

HD 

IBEPI 

IHD 

INPI 

IP/IPR 

European Strategic Cluster Partnerships 

EU International Property Office 

EU Business centres in Vietnam  

Frequently asked questions 

Helpdesk 

Ibero-American Industrial Property Programme 

international helpdesk 

National Institute of Industrial Property (IP office France) 

Intellectual Property/Intellectual Property Rights 

ITA IPR Desks 

Italian Trade Agency Intellectual Property Rights Desk 

 
 
 
KPI 

LA 

LAHD 

key performance indicator 

Latin America 

Latin America IPR SME Helpdesk 

MERCOSUR 

Mercado Común del Sur (Southern Common Market95) 

MS 

SEA 

SEAHD 

SME(s) 

Member States 

South-East Asia 

South-East Asia IPR SME Helpdesk 

small and medium-sized enterprise(s) 

95 Members: Argentina, Brazil, Paraguay, Uruguay; Venezuela suspended 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
1.  INTRODUCTION  

The  measure  "COS-WP2014-2-05  China,  Latin  America  and  South-East  Asia  IPR  SME  Helpdesks" 
within the COSME programme96 set up the International Intellectual Property Rights SME Helpdesks 
(IHD) with the aim to offer free information and advice services on relevant Intellectual Property (IP) 
issues to European Union (EU) SMEs doing business in China, Latin America (LA) and South-East Asia 
(SEA).  

The aim of this evaluation is to check to what extent the IHD services have reached their objectives in 
terms of relevance, effectiveness, efficiency, coherence and EU value added. The present evaluation 
supports  the  mid-term  review  of  the  Competitiveness  of  SMEs  (COSME  programme  and  the 
discussions on the future multiannual financial framework. 

The results of this evaluation will also help decide on how best to provide IP support with different 
types of partners in a more efficient and effective ways.  

The evaluation covers 28 months of the IHD activities in China and SEA and 22 months for LA i.e. the 
years since  the  pre-COSME  activities of the  IHD (China from 2010, SEA and LA from 2013)  and the 
current COSME grants i.e. from 2015 until its situation is captured in April 2017.  

2.  BACKGROUND to the IHD measure 

The  IHD  initiative  found  its  roots  in  the  European  policy  relating  to  research  and  SMEs  support, 
industrial  policy,  Intellectual  property  Right  (IPR)97  policy  and  innovation  policy.  Before  the 
initiative started,  the  landscape was  a more limited exploitation of international opportunities and 
innovation prospects both in- and beyond the Single Market. SMEs usually received help in relation 
to  IPR  via  their  national  export  agencies,  EU  MS  support  administrations;  EU  and  MS  trade 
representations, chambers of commerce and in-site via bilateral chambers of commerce or EU and 
MS trade representations.  Against  this framework, the  first European IPR Helpdesk  was  created in 
1999. The aim was to assist EU SMEs and beneficiaries of EU-funded programs which frequently do 
not  have  the  knowledge  and  resources  to  protect  their  intellectual  assets  through  Intellectual 
Property Rights (IPR)  and build their competitiveness effectively around it,  and need to reduce the 
risk of their intellectual assets being used by competitors or, if protected, being infringed; in dealing 
with IPR matters.  

Next,  in  the  framework  of  increasing  globalisation,  to  support  EU  SMEs  both  protect  and  enforce 
their IPR while trying to enter and grow in third countries in particular on the Chinese98, SEA and LA 
markets, the IHD, were established. The three regions targeted by the action comprised high-growth 
markets  where  EU  businesses,  and  especially  SMEs,  could  find  many  expansion  opportunities.  The 

96 Competitiveness of SMEs (COSME) programme 
97‘Intellectual Property Rights’ are used throughout this document in the broad sense, encompassing both so-called ‘formal’ 
means of protecting such rights (e.g. patents, trademarks, designs, geographical indications, copyrights) and ‘informal’ means 
(contractual  and  licensing  conditions,  management  of  business  operations  so  as  to  protect  trade  secrets,  planning  business 
participation in trade fairs, etc.) ‘Enforcement’ of these rights refers to any applicable means of enforcement such as through 
the  civil  or  criminal  judicial  systems,  administrative  means,  customs  authorities,  or  contractual  means,  as  available  in  the 
relevant European, Member State, or third countries' jurisdictions. 
98 
in 
important 
https://euipo.europa.eu/ohimportal/en/web/observatory/mapping-the-economic-impact) 

the  most 

counterfeit 

products 

player 

which 

origin 

(see 

the 

of 

is 

 
 
                                                           
initial  experience  with  the  China  IPR  SME  helpdesk  (CNHD)  run  as  a  pilot  until  mi-2008  and 
continued under the Competitiveness and Innovation Programme (CIP)99 from December 2010. The 
ASEAN and MERCOSUR HDs started under CIP100 in 2013. This help was however limited regionally 
and  in  scope.  The  CNHD  covered  mainland  China  while  the  MERCOSUR  HD  was  limited  to  the 
MERCOSUR  countries  and  Chile.  The  number  of  materials  that  was  produced  and  the  range  of 
activities  were  also  more  limited.  Under  the  CIP  programme,  the  European  Commission  (EC)  was 
responsible for the implementation of the China, ASEAN and MERCOSUR helpdesks.  

Since 2015, the IHD project is funded by the EU via the COSME programme. The COSME programme 
takes  a  more  targeted  approach  and  is  aimed  at  improving  the  business  environment  to  enhance 
competitiveness by  focusing  on  framework  conditions,  entrepreneurship,  and  access  to  finance 
and/or  markets.  It  is  the  Executive  Agency  for  Small  and  Medium  Enterprises  (EASME)101  that  is 
entrusted  by  the  EC  with  the  implementation  of,  inter  alia,  the  supervision  of  the  IHD.  The  new 
version of the CNHD covers Hong-Kong, Macau and Taiwan. Similarly, the MERCOSUR grew up into 
the LAHD, covering more countries. Under the COSME programme the IHD were asked to develop a 
more  customised  approach  including  e.g.  sectorial  publication.  In  2015102,  a  common  web  portal 
started to provide a single-entry point for all three regional entities of the IHD. 

The mission to the IHD set in COS-WP2014-2-05 was to 

1.  Set-up and operate the IHD 
2.  Provide advice and support services to EU SMEs facing IPRs difficulties in or arising from 

China, SEA and LA  

3.  Maintain the  link with intermediaries and stakeholders including the Enterprise Europe 
Network (EEN), industrial associations, and pan-European business support organisations 
in the target regions 

4.  Adapt  the  work  to  the  target  group  on  demand  incl.  by  developing  a  categorisation 

according to the sector and geographical area to which users pertain 

Each IHD has in particular the specific objectives to: 

1.  reduce  the  risk  of  counterfeit  products  entering  the  European  market  from  the 

concerned Markets;  

2.  protect  SME's  valuable  IPR  before  entering  and  while  doing  business  in  the  concerned 
markets  (e.g.  developing  the  market  there  for  their own  products  or  when  following  a 
company 
they 
a supplier when it enters China, LA and SEA); 

traditionally 

which 

have 

to 

been                                                                                                                                        

3.  help SMEs to find and constructively work with local administrations or service providers 

to enforce their IPR protection 

These specific objectives had to be achieved through a number of activities  

1.  Provide 1st line advisory services on IPR protection and enforcement free of charge 

99 For China the CIP work programme 2010 (ENT/CIP/10/B/N02C00) and 2013 (ENT/CIP/13/B/N02C00) 
100 For LA and SEA the CIP work programme 2012 (ENT/CIP/12/B/N04C00) 
101 https://ec.europa.eu/easme/  
102 see CNHD Annual Work Plan 2015 

 
 
  
                                                           
2.  Develop and publish background materials (incl. publications, factsheets) on IPR for EU 

SMEs  

3.  Provide training to EU SMEs on IPR protection and enforcement 
4.  Monitor national IP policies and legislation 
5.  Provide partnering services 
6.  Communicate and disseminate including by hosting and operating a single multilingual 

web portal  

7.  Put in place joint actions and other innovative services to EU SMEs 

 
 
 
 
Objectives of the IHD 

Strengthen the 
competitiveness of 
SMEs 

Promote the 
creation and growth 
of SMEs  

By improving access to 
global markets 

Reduce the risk of 
counterfeit 
products entering 
the EU market 
from the 
concerned 
markets 

Protect SMEs 
valuable IPR before 
entering and while 
doing business in the 
concerned markets  

Help SMEs find and 
constructively work 
with local 
administrations or 
service providers to 
enforce their IPR 
protection 

Main 
objectives of 
the overall  
COSME 
programme) 

Specific 
objectives of  
IHD 

Specific 
actions  
(IHD) 

Provision 
of 1st line 
advisory 
services 
on IPR 
protection 
& 
enforceme
nt 

Developm
ent & 
publicatio
n of 
backgrou
nd 
materials 
on IPR for 
EU SMEs 

Provision 
of 
training 
to EU 
SMEs on 
IPR 
protectio
n & 
enforcem
ent 

Monitori
ng of 
national 
IP 
policies 
& 
legislati
on 

Provision 
of 
partneri
ng 
services 

Communic
ation & 
disseminat
ion 
including 
the 
hosting & 
operation 
of a single 
multi-
lingual 
web portal 

Joint 
actions & 
other 
innovativ

e                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          

services 
to EU 
SMEs 

Set-up and 
operates IHD 

Operate the 
IHD 

Provide 
support to EU 
SMEs 

Maintain link 
with 
intermediaries 
& stakeholders 

Adapt work 
to the target 
group on 
demand 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3.  STATE of PLAY 

3.1.  The functioning of the IHD and to what extent they meet the requirement set in COSME 

  
The  IHD  started  function  in  2008  in  China  with  the  CNHD,  followed  by  the  ASEANHD  and 
Mercosur HD in 2013. The latter grew up into the LAHD in 2015. One consortium implements each of 
the  regional  entity  of  the  IHD.  There  are  three  consortiums  in  total.  The  LAHD  consortium  has  a 
stronger academic component  than the  two other entities. The  IHD currently offer, free of charge, 
first-line support in the area of IPR to EU SMEs involved in business activities in or relating to China, 
LA and SEA.  

  
Each  regional  entity  of  the  IHD  provides  a  helpline,  trainings,  publications,  a  website,  and 
partnering services, monitors national IP policies and legislation, and cooperates with other existing 
organizations  and  institutions  which  deal  with  IPR  protection  in  China103,  SEA104  or  LA105.  The 
cooperation  consists  in  providing  complementary  services  and  signposting  users  to  each  other 
depending on their needs.  

It is difficult to quantify the amount of SMEs benefiting from the IHD services. However, the quantity 
of  users  registered  for  the  helpline  gives  some  indication  on  the  quantity  of  SMEs  reached  by  the 
IHD. In April 2017 the IHD had registered 3154 helpline users (1347 for China; 590 for SEA and 1217 
for LA).  

Users are mainly  SMEs:  75%  of the survey's respondents were  SME  and  the IHD members confirm 
that they are in daily contact essentially with EU SMEs. On a closer look at the survey's respondents, 
SMEs from the manufacturing branches stand out as sector (31%), while the second largest sector of 
respondents  are  service  providers  of  different  kinds  (i.e.  Legal,  business/management  services  or 
R&D consultancy firms). 

The IHD operate under a grant agreement106 signed with EASME, which handles the technical 
  
and financial management. The EC provides general policy guidance. The IHD measure is steered with 
the help of around 50 different KPIs. Based on the performance reported to EASME in April 2017, the 
table below shows selected KPIs and the main achievement of the IHD as of that date. At that time, 
there were eight months left to reach the indicated goals (i.e. less than 30% of the contract time left). 

  

Table 1: IHD Main achievements, in April 2017 

CHINA 

SEA 

LA 

Number of registered 
helpline users  

1347 

590 

1217 

103 e.g. with the EU SME Centre  
104e.g.  with  the  EU  Business  Avenues,  EU  Business  centres  in  India  (EKONID),  Malaysia  (SEBSEAM),  Philippines, 
Thailand (EABC) and Vietnam (EVBN), as well as all European and bilateral chambers of commerce in SEA 
105e.g. with European and bilateral chambers of commerce in LA, MSs' and EU delegations, related associations, and other 
EU network (e.g. ELAN) 
106 i.e. 2015 Grant agreement for the China IPR SME HD; 2015 Grant Agreement for the SEA IPR SME HD and 2015 Grant 
agreement for the LA IPR SME HD including the Original Grant Agreement; Annexe 1 to original Grant Agreement and the 
Amended Grant Agreement 

 
 
  
                                                           
Number of enquiries to be 
answered 

Trainings modules to be 
developed 

Training sessions to be 
delivered 

Webinars to be delivered 

Number of training 
participants 

Number of partnering  
events attended 

1509 

652107  

687 

20 

125 

37 

11 

60 

38 

25 

60 

6 

7682 

3578 

1655 

78  

38 

0 

Number of visits  to the 
website 

119.692 

32.479 

20.000 
(estimated) 

New guides & other 
publications 

Articles published 

Source: EASME Policy Reports 

20 

239 

32 

230 

10 

N/A 

  
In  April  2017,  the  state  of  play  was  that  the  helplines,  overall,  experienced  high  levels  of 
demand,  with  the  number  of  registered  users  and  enquiries  exceeded  in  two  regions,  and  despite 
related challenges reported by the LAHD108. The websites will in all likelihood not be used as much as 
expected  (in  terms  of  page  views  and  visitors),  judging  by  the  data  available.  No  more  than  50% 
trainings had been organised and only one IHD entity had held more than 50% of the webinars. This 
is  also  reflected  in  an  overall  low  number  of  participants  in  the  training.  For  publications,  the 
relevant KPI seems slightly more positive than for the training.   

  
As can be seen from the Figure below, out of the range of services offered by the IHD, the 
website  and  the  publication  are  the  most  used.  More  than  50%  of  the  users  surveyed  have 
occasionally  visited  the  website,  10%  visit  it  frequently.  Around  45%  have  used  the  publication 
offerings at least occasionally, and around 10% use them frequently. The helpline services are ranked 
in  third  position  as  regards  the  frequency  of  use.  Around  a  quarter  of  the  respondents  have  used 
them occasionally.  

107 770 in June 2017 as reported in the Progress Report of the SEAHD 
108  A  challenge  that  occurred  recently  was  that  the  LAHD  faced  the  situation  of  a  poor  provision  of  web  services.  This 
resulted notably in having no helpline management system available since the beginning of May  2016, in a long response 
time of months in most of the cases, in the loss of website statistics after the website transfer in August 2016, and in a delay 
in the deliverance of dissemination materials, requiring overall much extra effort from staff. 

 
 
                                                           
  

Figure 18 Usage frequency of different types of services offered by the HDs, SMEs in absolute numbers, overall

Source: Stakeholder's consultation (n=71) 

  
  
  
From a regional perspective, the online training offerings appear as more frequently used by 
SMEs in SEA and China, whereas the figures in LA are significantly lower (around 20% occasional or 
frequent users versus 32% and 45%, respectively). Some intermediary organisation reported indeed 
webinars organised by the LAHD with a low  number of participants. It  appears also that  the  direct 
training offerings of the CNHD and the SEAHD are used more frequently than in LA (over 30% of the 
respondents  using  in  Asia  at  least  once  against  only  25%  in  LA).  The  number  of  training  events 
organised in China is the highest. The SEAHD has, on the one hand, organised fewer training events 
than the LAHD. However, these events have reached a higher total number of participants.  

  
Use of the helpline appears as more frequent in Asia than in LA: 70% of the Chinese and 50% 
of the SEA HD survey' respondents have used the helpline at least once. In LA this number drops to 
around 38%.  
  
Figure 19 Usage frequency of selected services offered by the HDs, by region

Source: Stakeholder's consultation n=24-36 

1%4%1%11%10%14%24%24%35%44%10%20%34%24%27%72%49%39%28%17%3%3%1%1%3%0%10%20%30%40%50%60%70%80%90%100%Direct training sessions,workshops, SME clinicsOnline training modules(webinars, etc.)The helpline serviceThe publications availableon the websiteThe website in generalHow frequently have you used the following services of the IPR SME Helpdesks?FrequentlyOccasionallyOnceNeverDo not know4%4%3%4%4%6%13%21%19%17%21%28%21%42%28%8%13%11%21%25%42%29%17%14%71%58%67%63%46%31%50%33%50%4%4%4%4%3%0%10%20%30%40%50%60%70%80%90%100%LASEACNLASEACNLASEACNDirect trainingsessions,workshops, SMEclinicsThe helplineserviceOnline trainingmodules(webinars, etc.)How frequently have you used the following services of the IPR SME Helpdesks?FrequentlyOccasionallyOnceNeverDo not know 
 
 
 
 
 
3.2. 

Previous evaluations and reports 

This is the first evaluation of the three regional entities of the IHD. However, the CNHD was already 
evaluated once. 

According  to  the  assessment  of  the  CNHD  that  was  conducted  in  2010109  by  an  external 
  
contractor,  among  SMEs  users,  75%  believed  that  the  CNHD  is  needed  and  at  least  60%  that  it 
provided good services. Face-to-face events, training and workshops generate more added-value.  

However, 40% of users expressed a certain degree of dissatisfaction, which might have been linked 
to the difficulties they face to understand the content of the CNHD offer. Future help should focus on 
enabling  SMEs  understand  and  comply  with  Chinese  administrative  process  including  by  putting 
them in contact with the relevant organisations/person. The acknowledged high-quality information 
services could benefit from a categorisation according to the sector and geographical area to which 
users pertain; increasing the range of materials provided to SMEs that have never been confronted 
before with IPR issues, with information produced in Chinese and in relation to the situation in China. 
A list of lawyers in China could help with infringement issues. The users are very satisfied with the 
high  quality  of  the  experts  used  in  China,  but  would  appreciate  the  inclusion  of  a  commercial  and 
in  Europe  to  develop  relationships  with 
multicultural  profile,  and  a  consistent  presence 
intermediaries at EU level.    

  
More  generally, building relationship with intermediaries  by notably increasing presence  in 
trade  fairs  and  cooperation  with  trade  missions,  as  well  as  sustaining  this  relationship  by  notably 
increasing the  training for intermediaries' staff would help raise  awareness amongst  SMEs  and the 
visibility of the CNHD.  

A LAHD110 conducted from November 2015 to March 2016 with a total of 344 LA SMEs respondents, 
distributed  across  most  of  the  LA  countries  analysed  that  efforts  should  be  focused  on  providing 
tailor made actions in the form of helpline support and training preferably via Webinars, employing a 
jargon-free language,  tailor-made  for an SME and ideally to its  sector; providing access to material 
such  as  fact  sheets  or tips,  videos  and  case  studies, and  complementing  trainings  with  networking 
activities  like  meetings,  B2Bs  where  SMEs  will  have  the  possibility to connect  with  each other  and 
with other intermediaries dealing with innovation as well as universities, technology and innovation 
centres, chambers of commerce, trade associations, etc. in strong synergy with key actors in the  IP 
context including the LA countries’ National Patent Offices. 

On  the  basis  of  the  2010 assessment  of  the  CNHD  and  of  the  Pilot  Study  in  LA,  under  the  COSME 
programme,  the  IHD  were  asked  to  develop  a  more  customised  approach  including  e.g.  sectorial 
trainings or publication.    

Previous  evaluations  or  studies  have  also  been  carried  out  by  the  Commission.  In  relation  to  the 
same activity but in a distinct geographical area, the evaluation of the European IPR Helpdesk111 run 
by an external contractor in 2013, concluded that some aspects of the services had to be improved in 
particular that a greater emphasis should be put on the promotion of the EU Helpdesk services, via in 

109 2010 assessment of the China IPR SME HD 
110Seehttp://www.latinamerica-ipr-helpdesk.eu/sites/default/files/la_ipr_sme_helpdesk_baja_pilot_study.pdf  
111 Report on the evaluation of the European IPR Helpdesk published in 2014 

 
 
 
                                                           
particular the implementation of a clear and effective marketing strategy and Communications Plan, 
the  development  of  closer  relationship  with  multipliers,  and  a  promotion  within  the  Commission 
services;   that the   possibility of providing greater flexibility within the  contractual form, while  still 
complying  with  procurement  law  needed  to  be  explored  further,  streamlining  the  reporting 
requirements  of  the  Helpdesk  contract,  including  in  the  report  information  on  performance 
indicators. 

In addition, the IHD themselves carried out an own Internal impact assessment exercise in 2016112. 

4.  METHODOLOGY 

The  evaluation  of  the  IHD  was  conducted  between  2  December  2016  and  January  2018  and  was 
steered  by  the  members  of  an  Inter  Service  Steering  Group  involving  various  services  of  the 
Commission.  It  has  been  partially  supported  by  an  evaluation  study  i.e.  "Support  Study  for  the 
Evaluation of the International Intellectual Property Rights SME Helpdesks" prepared by an external 
contractor113.  The  contractor  was  primarily  asked  to  get  stakeholders  views  (through  surveys  and 
interviews) on the functioning of the IHD and on the impacts on the objectives in terms of relevance, 
effectiveness, efficiency, coherence and value added and provide an analysis of their replies; and to 
analyse the coherence of the IHD with other COSME, EU and/or MS SME support measures, as well 
as other relevant programmes contributing to the internationalisation of European businesses and 
their competitiveness in China, SEA and LA. Both the  external study and the current  staff working 
document of the Commission services have been reviewed by three independent experts selected 
among  a  list  of  five  proposed  by  the  external  contractor.  Their  views  on  the  relevance  and 
coherence of the IHD initiative can be summarised as follow:  

On substance, the three experts limit their analyse of the relevance aspect to analysing whether the 
IHD  initiative  can  still  be  considered  relevant  for  achieving  the  objective  of  helping  SMEs  dealing 
with IP matters in the relevant territory. One expert applying a systemic approach and considering 
the Chinese angle only considers that at overall level, the EU IP system is obsolete. Because of the 
sheer market size and relevance for value chains, companies are forced to become active in China, 
and  to  cope  with  detrimental  policies  and  regulations  that  lead  to  actually  unwanted  know-how 
flows. Measures like the IHD initiative can at best slow this trend down but cannot help in avoiding 
it. A second expert views the initiative as overall still relevant for achieving the desired objectives, 
also in the context of China which has shown considerable changes in its IP system. At micro level 
and  under  the  assumption  that  the  IHD  would  be  largely  functioning  as  intended,  the  first  (China 
focussed) expert, still views the IHD initiative as not relevant. Reasons put forward are that SMEs are 
mostly  not  in  a  position  to  enter  China.  IP  issues  apart,  these  companies  would  find  it  difficult  to 
enforce  their  rights  because  of  lack  of  resources  particularly  financial  ones.  Moreover,  the  IPR 
system  conceived  in  Europe  in  the  18th  century  is  outdated.  It  cannot  match  current  shorter  and 
shorter innovation cycles. As a result it limits innovation. Innovation requires increasingly permeable 
markets. The second expert, a patent attorney, views SMEs from big EU countries as less in need of 

112 Internal impact assessment exercise carried out by the IHD themselves in 2016 
113 U:\IPR Helpdesks\02. International helpdesks\01. Intl HD 2017 evaluation\Technopolis Reports on IPR evaluation\Eval-
IPRHelpdesks_final-26Jan2018.pdf 

 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                                                           
the IHD services because they benefit from a very dense and professional network of IP advisors. His 
personal  practical  experience  (in  Germany)  shows  a  lack  of  visibility  of  the  IHD.  His  clients  hardly 
mention  the  IHD  services,  while  his  colleagues  have  usually  not  heard  about  the  IHD  (at  least  in 
sufficient  details).  The  third  expert  considers  that  the  little  use  and  take–up  of  the  IHD  services, 
acknowledged  by  the  KPIs,  could  indicate  a  limited  relevance.  This  expert  focuses  on  the 
methodological aspect of the analysis and considers that most  elements presented in the external 
study  and  the  staff  working  document  of  the  Commission  services  are  available  for  an  adequate 
assessment  of  relevance  of  the  IHD  initiative.  However,  the  analysis  of  the  quantity  of  SMEs 
benefitting  from  the  service  should  be  expanded  to  a  more  detailed  analysis  of  the  type  of 
enquires/type of topics covered – which could inform an assessment of the nature of needs which 
SMEs have in relation to IP issues.  

Having  acknowledged  their  lack  of  expertise  in  general  EU  policies,  the  experts  analyse  that  the 
coherence with other EU policies could be limited due to a lack of collaboration with the European 
Patent  Office  resulting  from  an  originally  present  personality  having  gone,  while  the  fact  that  the 
China  IPR  SME  helpdesk  does  not  differentiate  Taiwan  and  mainland  China  as  different  territories 
would be a clear sign of in-coherence. From a methodological point of view, overall, the data which 
is  presented  in  the  external  study  and  the  staff  working  document  provides  a  good  basis  for 
assessing coherence of the IHD initiative. The finding of the staff working document that IHD should 
ensure  better  communication  with  other  COSME  initiatives  and  improve  collaboration  seems 
relevant  given  the  findings  on  coherence.  However,  given  that  all  the  initiatives  considered  are 
funded and managed by the EC, there would be benefit in considering how the EC could promote 
and facilitate collaboration.114 

 The following evidence has been used in the evaluation: 

  COSME Regulation (EU) No 1287/2013115 
  Competitiveness and Innovation Programme (CIP)116  
  Competitiveness of SMEs (COSME) programme117  
  For  China, 

the  CIP  work  programme  2010 

(ENT/CIP/13/B/N02C00)119  

(ENT/CIP/10/B/N02C00)118  and  2013 

  For LA and SEA, the CIP work programme 2012 (ENT/CIP/12/B/N04C00)120 
  2012 Grant Agreement for the SEA IPR SME HD121  
  2013 Grant Agreement for the Mercosur IPR SME HD122  
  2015 Grant agreement for the China IPR SME HD123 
  2015 Grant Agreement for the SEA IPR SME HD124 

114 See "Expert Group assessment on the evaluation of the EU international IPR SME helpdesks (IHDs)", Synthesis report 
prepared by Alfred Radauer under the following link:  U:\IPR Helpdesks\02. International helpdesks\01. Intl HD 2017 
evaluation\Experts opinion\ExpertGroup-Report-final.docx   

115http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1287  
116 http://ec.europa.eu/cip/  
117 https://ec.europa.eu/growth/smes/cosme_en  
118 http://ec.europa.eu/cip/files/docs/eip2010_wp_annex_indicators_en.pdf  
119 http://europski-fondovi.eu/sites/default/files/dokumenti/eip-2013-work-programme-final_en.pdf  
120http://ec.europa.eu/cip/files/cip/docs/eip-2012-work-programme-consolidated-version-after-2nd-revision-adopted-26-
november_en.pdf  
121 see 2012 Grant Agreement for the SEA IPR SME HD    
122 see 2013 Grant Agreement for the MERCOSUR IPR SME HD 
123 see 2015 Grant agreement for the China IPR SME HD 

 
 
 
  
                                                           
  2015 Grant agreement for the LA IPR SME HD i.e. 

  Original Grant Agreement125 
  Annexe 1 to original Grant Agreement126 
  Amended Grant Agreement127 

  As  well  as  information  given  by  EASME  including  quarterly,  annual,  and  biannual  interim 
reports submitted by the three IHD entities, as well as Policy Officer (PO) reports and other 
deliverables. 

A number of different stakeholders were consulted through web surveys and 39 interviews. Separate 
ad-hoc survey were run for SMEs users of the IHD and potential SMEs user of the IHD located in the 
EU  and  already  doing  business  in  the  target  region(s)  or  considering  internationalisation.  The  first 
category was reached through each IHD, while the survey addressed to the potential users of the HID 
was circulated via intermediaries like, IP support services, embassies, chambers of commerce, etc.   

Other stakeholders consulted were: business representatives of SMEs like associations; chambers of 
commerce, industry associations, sectorial associations, etc.; national export promotion agencies of 
EU  MS  and  bilateral  EU  MS-China,  SEA  and  LA  business  associations;  experts/consultants/service 
providers on IP, SME and internationalisation; MS ministries steering the COSME programme and MS 
trade  counsellors  from  embassies  in  the  target  countries.    In  addition,  a  public  consultation  was 
carried on for the overall COSME programme128.  

It is important to note that the  data from the surveys come from a non-representative sample of 
SMEs.  Moreover,  despite  the  efforts  to  maximise  the  replies  to  the  surveys,  the  overall  number 
remained quite low: 70 replies for the users survey and only 41 for the non-users survey.  

The  limitations  of  the  dataset  collected  via  the  survey  was  balanced,  where  feasible,  through 
triangulation with complementary data sources, first of all the Key Performance Indicators reported 
for the evaluation period by each consortium to the Executive Agency for Small and Medium-sized 
Enterprises  (EASME)  and  secondly  interviews  with  the  helpdesk  personnel,  relevant  staff of the  EC 
namely  DG  GROW,  DG  TRADE  and  EASME,  SMEs,  intermediaries  e.g.  business  consultancies, 
stakeholders  and  similar  service  providers.  Other  sources  of  information  came  from  existing 

124https://myintracomm-
collab.ec.europa.eu/networks/COSMESI/Shared%20Documents/COSME%202014/Objective%202%20-
%20Access%20to%20Market%20-%20Code%20D/D-N205%20-%20IPR%20SME%20Helpdesk%20China%20-
%20South/Selected%20Projects/Grant%20Agreement-657914-SEA%20HD%202015-2017.pdf  
125https://myintracomm-
collab.ec.europa.eu/networks/COSMESI/Shared%20Documents/COSME%202014/Objective%202%20-
%20Access%20to%20Market%20-%20Code%20D/D-N205%20-%20IPR%20SME%20Helpdesk%20China%20-
%20South/Selected%20Projects/17_03_09_Grant%20Agreement-657846-LA%20IPR%20SME%20HD.pdf  
126https://myintracomm-
collab.ec.europa.eu/networks/COSMESI/_layouts/15/WopiFrame.aspx?sourcedoc=/networks/COSMESI/Shared%20Docume
nts/COSME%202014/Objective%202%20-%20Access%20to%20Market%20-%20Code%20D/D-N205%20-
%20IPR%20SME%20Helpdesk%20China%20-%20South/Selected%20Projects/17_03_09_Annex%201%20-
%20Description%20Of%20Action%20(part%20A).pdf&action=default  
127https://myintracomm-
collab.ec.europa.eu/networks/COSMESI/_layouts/15/WopiFrame.aspx?sourcedoc=/networks/COSMESI/Shared%20Docume
nts/COSME%202014/Objective%202%20-%20Access%20to%20Market%20-%20Code%20D/D-N205%20-
%20IPR%20SME%20Helpdesk%20China%20-%20South/Selected%20Projects/Amendment%20AMD-657846-
4.pdf&action=default  
128  Among  the  14  additional  contributions  provided  by  certain  contributors  in  the  framework  of  this  consultation,  four 
mentions the IHD.  

 
 
 
                                                                                                                                                                                     
materials i.e. for each regional entity of the IHD, the grant agreement (defining in particular the KPIs), 
the  table  describing  the  work  in  terms  of  a  list  of  deliverables  and  KPIs  and  their  current  status 
(pending,  submitted or  approved),  the  Progress  Report,  the  Periodic  Report(s);  EASME's  three  "PO 
Assessment  Report"  assessing  the  first  Periodic  Report  of  each  regional  entity;  other  background 
documents i.e. agenda, minutes, mission report and action plan of the monitoring visits performed to 
Alicante  (LAHD)  and  Beijing  (CNHD);  handover  documentation  of  the  CNHD  funded  by  the  CIP 
programme;  and  the  results  of  an  internal  impact  assessment  exercise  carried  out  by  the  IHD 
themselves in 2016.  

While  the  interpretation  of  the  data  presented  needs  to  be  done  with  care  some  important 
conclusion could be drawn.    

The survey with users and non-users of the IHD was executed electronically between August 2017 
and September 2017. Information about the synopsis report on the consultations129, as well as the 
survey respondent recruitment, response rates and respondent characteristics130 and the survey 
questionnaires131 can be found as annexes to the Study. In the same way, the Study provides 
information about the interview programme, interview guidelines, and interviews synthesis132.  

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

In  accordance  with  the  Better  Regulation  guidelines133,  the  evaluation  has  to  determine  on  an 
evidence-basis whether the actual performance of the IHD has been at least relevant given the needs 
and  its  objectives,  effective  and  efficient,  coherent  both  internally  and  with  other  EU  policy 
interventions, and has achieved EU added-value. To guide this analysis, a set of evaluation questions 
have been asked in relation to each of the five criteria. They are mentioned below. 

5.1.  RELEVANCE 

A key consideration for the evaluation is whether the results obtained by the IHD are still regarded as 
useful and pertinent to the current needs of users. Evidence collected from existing studies as well as 
the result of the evaluation show that the IHD serves the most relevant needs of SMEs in the area.    

5.1.1. 

Is the need to protect IP still relevant? 

Overall IP remains a major part of the value of companies, and IPR an important element for a 
significant amount of businesses: several recent studies134, among which the OECD/EUIPO 2016135, 
show that as much as 5.1% of EU imports in 2013 was in counterfeit and pirated goods, amounting to 
Euro 85 Billion. China is the Number one provenance economy of counterfeit goods entering the EU 
while SEA (Singapore 9th, Thailand 15th) and LA countries appear in the top-15 list. Similarly, the 2017 
edition of the special 301 report by the United States Trade Representative – a report on the state of 

129 see Study, Appendix A 
130 See Study, Appendix B 
131 See Study, Appendix F 
132 See Study, Appendix C, D and E 
133 Better Regulation guidelines 
134 See, for example, OceanTomo (2016): Intangible Asset Market Value Study, http://www.oceantomo.com/intangible-
asset-market-value-study/; or: WIPO (2017): Intangible Capital in Global Value Chains, 
http://www.wipo.int/edocs/pubdocs/en/wipo_pub_944_2017.pdf  
135 OECD/EUIPO (2016): Trade in Counterfeit and Pirated Goods – Mapping the Economic Impact. 

 
 
 
                                                           
IP protection and enforcement in US trading partners around the world – places 11 countries on the 
top  priority  watch  list  for  issues  in  relation  to  IP  enforcement.  This  list  includes  China;  Indonesia, 
Thailand,  Argentina,  Chile,  and  Venezuela,  i.e.  six  countries  that  are  addressed  by  the  IHD.136  The 
2016  "European  SME  Exporting  Insights  Study",  a  survey  of  2016  of  12 815  SMEs,  conducted  by 
logistics firm UPS, in eight major EU countries shows that the share of exporting SMEs in the EU has 
steadily increased between 2014 and 2016. There may be as many as 1 million active exporting SMEs 
in Belgium, France, Germany, Italy, the Netherlands, Poland and the UK137. Similary, a survey of KfW 
Group  in  2016  (n=600)  revealed  that  about  50%  of  German  SME  turnover  in  the  manufacturing 
industries  is  accounted  for  through  export  activities,  and  “…89%  of  manufacturing  SMEs  sell  into 
foreign markets, either directly or via their exporting customers.”138. Hence there is a strong rationale 
for still supporting EU firms in enforcing their IP rights in these regions. In addition, all these studies 
point  out  a)  that  there  is  still  a  problem  with  IP  infringements,  in  particularly  in  third  countries 
targeted  by  IHD  and  that  b)  a  sizable  number  of  EU  SMEs  is  likely  to  face  in  particular  while 
exporting139.  

  
A large majority of the users surveyed (80%) consider the assistance that the IHD provides 
is essential both when planning to expand to a third country and once they have already entered 
the market. This could indicate that support for IP management among SME is still a relevant issue 
in the context of their international activities while still operating in the EU and planning to expand 
outside Europe and once they have entered a third country. 

Five  of  the  seven  specific  actions  of  the  IHD  are  oriented  towards  satisfying  the  SMEs’  need  for 
information  and  advice.  The  results  of  the  survey  rank  as  first  priority  the  "information  on  the 
difference  formal  ways  to  protect  IP"  (59%),  followed  by  "information  on  registration  and  other 
procedures" (41%). This can be seen from the following figure.  

Figure  20 

  Share  of  users  with  needs  for  advice  and  support  on  specified  topics,  by  HD, 

in  %

Source: Stakeholder's consultation 

136 Office of the United States Trade Representative (2017): 2017 Special 301 Report. 
137 UPS (2016): 2016 European SME Exporting Insights Study 
138 Abel-Koch (2016): SMEs’ value chains are becoming more international – Europe remains key 
139even if the cited export analyses of SMEs might indicate that it is only a smaller share of the European SME population 
that  may  be  affected  by  this,  the  actively  exporting  SMEs  with  or  without  foreign  direct  investments  represent  still  a 
significant amount.  

79%39%21%18%33%18%6%9%75%58%8%25%50%25%25%8%25%75%50%0%0%25%0%0%10%20%30%40%50%60%70%80%90%100%Informationon thedifferentformal waysto protect IP(n=4-33)Informationonregistrationand otherprocedures(n=4-33)Assistancewith licensing(n=4-33)Assistancewithconfidentialityagreements(n=4-33)ImplementingIPmanagementstrategies inthirdcountries(n=4-33)Assistance insituations ofinfringementsof our IP byothers (n=4-33)Assistance insituations ofallegedinfringementsof others IPby ourselves(n=4-33)Mediation andalternativedisputeresolution(n=4-33)For which IPR issues have you needed advice and support?West/NorthSouthEast 
 
 
 
                                                           
Interviews  corroborate  this  finding  as  well.  Overall  the  need  to  get  reliable  information  to  protect 
and manage adequately IPR in the different targeted markets is the most important mission of the 
IHD for SMEs users. This comes over operational assistance like assistance with licensing (16% for the 
three regions) or confidentiality agreement (15%) or in situations of infringements. The IHD are also 
not  especially  asked  for  support  in  implementing  IP  management  (28%  for  all  regions)  strategies 
(except for the LAHD) nor for mediation (6%). This shows that the focus on provision of IP expertise 
offering  a  relevant  information  and  support  adapted  to  the  targeted  markets  (via  the  helplines, 
trainings, publication and website services) remains pertinent.  

The  survey  gives  also  interesting  information  about  how  important  the  EU  SME’s  users  view  the 
specific features of the IHD services.  

Figure 21 Importance of various aspects of the services provided in the view of users 

Source: Stakeholder's consultation. n=56-58. 
  

Overall,  user  and  non-user  SMEs  particularly  value  the  importance  of  having  local  support  outside 
the EU (around 60% rate it as "very important"), the quality of advice (more than 70% of the users’ 
sample), and the depth of information of the services (around 50% users and non-users). EU SMEs 
using the IHD value also highly the fact that their services are offered free-of-charge (more than 50% 
rate  it as "very important").  Above  all,  these  findings confirm that all the  characteristic features  of 
the  IHD  offer,  as  initially  thought,  are  regarded  as  important  and,  in  a  high  proportion  "very 
important, by around 90% of the surveyed SMEs. This is clearly a clue of relevance. 

Another indicator for a high relevance  of the IHD is the  finding that more than around 80% of the 
survey' respondents thinks that the  services  should be  extended to other areas of the world, India 
coming as first mentioned country.  

5.1.2. 

Should  the  scope  of  the  IHD  be  broadened  to  any  IPR  issue  potentially 
faced by EU SMEs (including litigation)? 

64%44%47%46%74%53%51%21%42%39%46%21%41%46%11%5%12%5%4%3%2%4%9%2%2%2%2%2%0%10%20%30%40%50%60%70%80%90%100%Having local supportin the regions/countriesExtent of the network(s)managed by the helpdesksInformation providedin a language that I masterSpeed of the replyQuality of the adviceIt is free of chargeDepth of the informationprovidedHow do you rate the importance of the following aspects of the business support services offered by the International IPR SME helpdesk?Very importantImportantNot importantDo not know 
 
 
 
 
  
Evidence  shows  that  there  is  an  interest  in  deepening  the  services  within  the  scope 
currently  offered.  In  general,  a  large  majority  of  SME  users  (more  than  80%)  responding  to  the 
survey  indicate  that  it  will  remain  of  special  importance  to  obtain  further  information  on  the 
different ways to protect IP, as well as further assistance with registration and other procedures. This 
matches the top two needs already expressed140. Between 70% and 80% of the survey' respondents 
consider  also  important  or  very  important  for  the  future  to  obtain  more  assistance  with 
confidentiality  agreement,  licensing,  alternative  dispute  resolutions  or  litigation  and  enforcement. 
Disparities appear between the different geographical areas. In particular, EU SME in China strongly 
claim  for  more  assistance  with  confidentiality  agreements  (95%),  while  litigation  and  enforcement 
issues, including alternative dispute resolutions, appear to be a of special interest in SEA (more than 
80%  of  the  replies).  From  the  interviews,  the  IHD  members,  based  on  their  experience,  see  an 
interest,  for  users,  in  benefitting  from  support  during  mediation  efforts  in  SEA  (where  it  could  be 
offered during trade fairs), and in China (where such service is, as a matter of fact, already currently 
provided).  This  evidence  shows  that  when  the  respondents  did  not  rank  these  items  among  their 
priority needs, they still value them for the future.   

  
Overall,  the  original  range  of  services  as  defined  in  the  framework  of  the  COSME 
programme seems to fulfil the needs of EU SMEs facing IPR difficulties in or arising from China, SEA 
and LA. Interviews do not specifically outline new services to be added to the current offer from the 
IHD. The demand focuses more on a deepening of the IHD expertise than on a broadening of their 
current range of services. 

Another  issue  is  whether  the  SME  focus  is  still  relevant.  The  target  group  of  the  IHD  is  SMEs 
according  to  the  EU  definition  and  firms  slightly  above  this  size  limit.  National  support  services 
usually do not have such a company size restriction. Question arises as to whether this target group 
definition limitation still makes sense. Interviews evidence that the IHD members consider the SME 
focus to be very relevant first because SMEs are the backbone of the EU economy, second because 
the  high  number  of  inquiries  and  participations  in  their  activities  shows  that  EU  SMEs  feel 
concerned. A possible inclusion of larger companies, which might miss expertise about new markets, 
is also voiced. On a closer look, the CNHD and SEAHD are aware of the SMEs specific needs and use a 
tailored  approach whereas  the  LAHD  considers  it  to some  degree  challenging  to  cater  for  SMEs  as 
they are very heterogeneous. SMEs tend to perceive the topic of IP as relevant only to larger firms, 
which, in itself, shows the need to make SMEs sensitive to it. Most stakeholders, especially similar 
service providers, argue that SMEs have a higher need since they tend to be under-informed about 
IP matters. Some chambers of commerce especially in SEA argue that it is usually larger companies 
who  have  high  investments  in  the  area  that  should  be  protected.  However,  these  companies  do 
usually have their own lawyers.  

To  conclude  on  this issue some  elements  need  to  be  further  analysed,  in  particular,  statistics  on 
the  quantity  of  SMEs  benefiting  from  the  IHD  services.  There  is  no  indicator  on  the  quantity  of 
SMEs  facing  IPR  difficulties  in  or  arising  from  China,  SEA  and  LA  that  the  IHD  have  reached.  The 
number  of  users  registered  for  the  helpline  only  gives  some  information  about  the  quantity  of 
SMEs reached by the IHD. In April 2017 the IHD had registered 3154 helpline users (1347 for China + 
590  for  SEA  +  1217  for  LA),  and  this  often  despite  the  absence  of  an  efficient  IP  tool  for  such 

140 See above comments made on Figure 3  

 
 
                                                           
registration. The CNHD and LAHD over performed their target, while the SEAHD achieved less due to 
a belated implementation of the helpline. With 1347 registered helpline users against 1000 targeted, 
China performs well. According to the European SME Exporting Insights Study 2015141, only around 
42.900 out of the 390.000 SMEs of seven large EU MS142 from four sectors143 export to China (which 
represents 11%). With 1347 SMEs registered as helpline users, the CNHD reaches around 3% of these 
EU  SMEs.  On  such  basis,  the  IHD  could  still  substantially  increase  the  number of  their  users  in the 
future.  

Conclusion on relevance: 

Evidence  shows that the  IHD have  been well designed to make  EU SMEs  aware  about  the need to 
take care of their IP rights before and after entering the targeted third countries markets. The IHD do 
this  by  providing  EU  SMEs  appropriate  information  and  encouraging  them  to  take  the  rights 
measures  to  protect  their  assets.  In  this,  the  IHD  initiative  potentially  contributes  to  the  overall 
objective of the COSME programme to strengthen competitiveness of EU SMEs. It could contribute as 
well to the specific objective of reducing counterfeit. For both objectives the IHD need to reach the 
critical mass of EU SMEs.  

5.2.  EFFECTIVENESS 

5.2.1. 

How effective the IHD action has been in achieving or progressing toward 
their objectives? 

The  COSME’s  programme  set  a  mission  to  the  IHD  which  require  the  implementation  of  specific 
actions: 
line  advisory  services,  trainings,  publications,  website,  partnering  services, 
communication and dissemination, joint action with other services to EU SMEs. 

first 

In  terms  of  general  assessment  of  effectiveness,  the  evidence  suggests  that  the  IHD  successfully 
achieved providing useful advice and support services to EU SMEs facing IPR difficulties in or arising 
from  China,  SEA  and  LA.  The  main  positive  effect  resulted  in  actually  raising  the  SME’s  awareness 
about the necessity to take care of their IP rights. 

According  to  the  survey,  over  80%  of  the  user  SMEs  responding  consider  the  IHD  overall  to  be 
"effective"  or  "very  effective"  in  raising  awareness  about  the  value  of  intellectual  assets.  70%  find 
them  effective  in  raising  awareness  of  the  necessity  to  secure  and  manage  IP  in  the  countries 
covered. 

The  effectiveness  analysis  focusses  on  the  results  that  the  IHD  have  been  delivering  during  the 
evaluation period in the light of the mission set by the COSME Programme. It should allow determine 
also to what extent these achievements have positive effects on the three specific objectives of the 
IHD initiative144.  

141 European SME Exporting Insights Study 2015  
142 i.e. Belgium, France, Germany, Italy, the Netherlands, Poland and the UK 
143 i.e. industrial manufacturing & automotive (IM&A), retail, high-tech and healthcare 

144 See above Table showing the Objectives of the IHD 

 
 
 
 
 
                                                           
The efficiency analysis considers first whether the IHD optimised the use of their resources given the 
obtained (quantitative and qualitative) results. 

Effectiveness of the IHD measure in achieving its mission 

a.  Objective 1: Operate the international IHD 

As  evidenced  from  their  website,  KPIs,  quarterly  reports  and  other  deliverables,  the  IHD  went 
operational. By  the  end of 2016, an IHD has been settled and offered to SMEs in Mainland China, 
Hong Kong, Macao, Taiwan; LA (covering Brazil and Mexico as well as other countries in the area like 
Argentina,  Bolivia,  Chile,  Paraguay,  Peru,  Uruguay  and  Venezuela,  and  some  central  American 
countries depending on EU business interest); and SEA (covering Indonesia, Thailand and Vietnam as 
well as other countries in the area like Brunei, Cambodia, Laos, Malaysia, Myanmar, the Philippines 
and Singapore). To offer such services the main project teams and physical contacts points in Beijing; 
Brazil,  Mexico  and  Chile;  Indonesia,  Vietnam  and  Singapore;  and  Belgian  and  Spanish  European-
based offices, started to operate in 2015, after appropriate staff recruitment and equipment bought.  

Its  implementation  experienced  however  some  slow  performance  in  certain  areas.  It  is  only  in 
October  2015  that  the  IHD  services  could  become  fully  available  in  SEA.  The  IHD  experienced  also 
some  difficulties  to  develop  activities  in  Brazil  and  Mexico  due  to  external  hindering  factors.  The 
most  critical  problem  that  hindered  the  implementation  was  the  absence  of  a  common  website 
during more than one year. It is only in March 2016 that a common website portal was developed.  

It  is  to  be  noted  that  despite  this  major  drawback,  all  key  services  i.e.  the  enquiry  helpline, 
publications on IPR-related issues, policy input, the provision of training to EU SMEs have however 
been delivered.  

b.  Objective 2: Provide advice and support services to EU SMEs facing IPRs 

difficulties in or arising from China, SEA and LA 

To answer this question, four points must be checked:  

1.  Did the IHD succeed in reaching the targeted EU SMEs facing IPR issues? 
2.  To what extent the targeted SMEs are aware of the existing IHD services? 
3.  Have SMEs users found the services that meet their needs? 
4.  How do the surveyed users assess the IHD initiative? 

1.  The IHD reach out the targeted EU SMEs and intermediaries 

As already mentioned,  the IHD progress reports do not provide data about  the quantity and the 
typology of SMEs that use their services. The only available indicator is the number of registered 
helpline  users.  It  is  reported  that  the  CNHD  and  LAHD  have  already  over  performed  their  target 
(1347 registered against  1000 targeted for the CNHD and 1217 registered against  800 targeted for 
the  LAHD).  Due  to  the  belated  implementation  of  its  helpline,  the  SEAHD  reached  only  590  users 
(1000 targeted) but should increase significantly this number.  

 
 
 
 
 
 
These results are encouraging and might increase considering the potential of EU SMEs facing IPR 
protection problems (as already seen above the CNHD reaches only around 3% of the EU SMEs that 
export to China).  

The recurrent  problem is that  it  remains complex to know precisely what kind of users call upon 
IHD services. 

The  survey'  respondents  gives  a  first  idea.  As mentioned  in  the  state  of  play,  the  largest group  of 
respondents (around 75%) is made up of companies having less than 50 employees.  

  
Among  these  EU  SMEs,  the  different  sub-sectors  of  the  manufacturing  branches  dominate 
the  sample.  31%  or  the  respondents  come  from  this  group.  Logically,  the  manufacturing  sector 
gathers most of the SMEs facing IPR issues. The second largest sector in the sample is represented 
by SMEs operating as service providers of "legal, business/management services" or "Engineering, 
R&D consultancy services". The two sectors represent cumulatively 35% of the sample. Such a high 
proportion is questionable. While the COSME Programme does not list law firms among the list of its 
beneficiaries but does not exclude them either145, it is dedicated to support entrepreneurs and their 
representatives. The primary objective of the IHD is not to provide advice to law firms, or  business 
consultancies  intermediaries.  When  such  intermediaries  are  involved,  the  link  between  the  service 
provider  and  SMEs  is  not  direct.  In  such  cases,  the  target  (i.e.  SMEs)  could  appear  as  somehow 
missed. On the other hand, this could show that the IHD offer their IP services to law firms, which in 
turn  act  as  multipliers  by  offering  to  their  final  clients  (i.e.  SMEs)  a  broader  service  including  IP. 
Interviews  among  the  IHD  members  and  a  panel  of  intermediaries  support  this  view.  The  role  of 
intermediaries could reveal problematic in case they would divert SMEs from the free of charge IP 
support services offered by the IHD. To avoid this negative impact, care should be taken to insist 
on the promotion of the IHD services to increase their visibility. This is all the more true considering 
that,  as  already  mentioned  (see  relevance  part),  there  is  a  gap  between  the  number  of  EU  SMEs 
registered  as  users  of  the  helpline  of  the  IHD  and  the  number  of  SMEs  facing  IPR  difficulties  in  or 
arising from China, SEA and LA that the IHD could potentially reach.  

Both the sample composition and the number of helpline registered users concur in showing that the 
IHD might to some extent have been effectively reaching the objective of providing IP support and 
advice to an adequate number of EU SMEs users. 

2.  The IHD do not get sufficient visibility 

  
The effectiveness of the IHD depends on how they reach out to their beneficiaries notably 
by promoting efficiently their services. The visibility of the  IHD is limited. In the  views of the  SME 
users responding to the survey, overall, around 50% find the IHD to be promoted very well or well. 
On the other hand, nearly 30% find the promotion to be insufficient and another 14% do not know 
how  well  the  IHD  were  promoted.  In  comparison  to  other  survey  responses,  these  numbers  seem 
overall low and could indicate room for improvement. This reveals particularly true in LA, where less 

145 ANNEX to the Commission Implementing Decision concerning the adoption of the work programme for 2014 and the 
financing  for  the  implementation  of  Programme  for  the  Competitiveness  of  Enterprises  and  small  and  medium-sized 
enterprises does not list law firms among the representatives, but does not exclude them either. 

 
 
 
 
 
 
                                                           
than 50% of the users surveyed judge the promotion to be very good or good. The lack of attendance 
to  partnership  events  reported  by  the  LAHD  could  corroborate  further  this  finding  (see  below 
"efficiency" part). Around 30% of all respondents think that the promotion of the IHD is insufficient. 
Some  interviews  point  out  that  this  could  be  related  to  the  fact  that  the  LAHD  is  managed  from 
Europe and not "on the ground" in LA, others link this to the fact that LAHD is the youngest, covers 
more  countries  and  does  not  benefit  from  local  political  support  as  much  as  other  regions  do  e.g. 
China. The interviewed stakeholders from the  EC confirm that the IHD have  difficulties  in reaching 
out to SMEs and creating sufficient visibility. Of even more concern is that among the sample of 43 
SMEs non-users surveyed, no one answer that they know the services of the IHD.  

It is interesting to see below the difference in perception of the IHD promotion between the group of 
SMEs and the group of ‘intermediaries’ (expert promotion agencies, business association, innovation 
support organisation, law firms, IP professionals). 
Figure 

22  Promotion  of 

view  of  user 

the  helpdesks 

intermediaries

and  user 

SMEs 

the 

in 

  

Source: Stakeholder's consultation=64 (user), n=27 (intermediary). 

  
Among  the  intermediaries  group,  more  than  80%  state  that  the  IHD  are  promoted  well  or 
very well. This suggests that the IHD promotion has more effect on intermediary organisations than 
on  the  SMEs  themselves.  As  such  the  intermediaries  appear  as  a  successful  alternative  way  to 
compensate a lack of direct contact with SMEs. However, since the main target group of the IHD 
are  EU  SMEs,  it  appears  pertinent  to  develop  communication  and  dissemination  to  gain  more 
visibility as well as to provide partnering services. 

3.  The  IHD  meet  the  EU  SMEs  needs  especially  by  raising  their  awareness  for  IPR 

protection 

  
With  the  objective  to  provide  effective  supports  to  EU  SMEs,  the  IHD  are  requested  to 
implement different specific actions such as the provision of first line advisory services, development 
of trainings, publications and of websites. The figure below provides a view on the effects of these 
actions.   

22%22%36%59%28%11%14%7%0%10%20%30%40%50%60%70%80%90%100%User SMEUser IntermediaryIn your opinion, how are the services of the International IPR SME Helpdesk promoted?Very wellWellInsufficientlyDo not know 
 
 
 
Figure 

6 

Effectiveness 

of 

helpdesks 

in 

the 

view 

of 

users 

across 

various 

aspects 

  

Source: Stakeholder's consultation. n=45-47. 

  
Over 80% of the user SMEs responding to the survey consider the IHD overall to be at least 
effective146 in raising awareness of the value of intellectual assets; 70% find them effective in raising 
awareness of the necessity to secure and manage IP in the countries covered. The IHD are viewed as 
less effective in solving concrete IP problems in the countries covered (just under 60% find them at 
least effective) and in developing capacities in SMEs to deal with IPR (just over 50%).  

These  results  make  sense  while  considering  the  most  important  needs  for  IPR  advice  and 

  
support ranked by the respondents in the survey as shown below. 

Figure 7 Share of users with needs for advice and support on specified topics, overall, in % 

  
  

Source: Stakeholder's consultation. n= 60 

  
The  parallel  display  of  the  two  above  Figures  tends  to  shows  that  the  IHD  are  effective  in 
catering the major need of the EU SMEs. EU SMEs first require being in a position of managing their 
IP  issues  in a specific  geographical area by accessing the relevant  information, as well as adequate 
support  services  and  advices.  The  IHD  appear  as  less  effective  on  other  aspects  that  the  survey' 

146 i.e. "very effective" or "effective"  

16%16%20%16%25%30%27%36%33%42%48%53%20%16%16%13%9%4%38%33%31%29%18%13%0%10%20%30%40%50%60%70%80%90%100%Building networks and relationswith experts and institutionsin the countries coveredBuilding long-term partnershipswith relevant EU networksDeveloping internal capacitiesin SMEs to deal with IP/IPRSolving concrete IP problemsin the countries coveredby thehelpdesksRaising awareness of the necessityto secure and manage IPin the countries coveredRaising awareness of thevalue of intellectual assetsIn your opinion, how effective is the International IPR SME Helpdesk as a mechanism to achieve each of the following objectives?Very effectiveEffectiveNot effectiveDo not know59%41%28%16%15%15%7%7%6%0%10%20%30%40%50%60%70%Information onthe differentformal ways toprotect IPInformation onregistrationand otherproceduresImplementingIPmanagementstrategies inthird countriesAssistancewith licensingAssistancewithconfidentialityagreementsAssistance insituations ofinfringementsof our IP byothersOtherAssistance insituations ofallegedinfringementsof others IP byourselvesMediation andalternativedisputeresolutionFor which IPR issues have you needed advice and support? 
 
 
 
                                                           
respondents  consider  less  important  such  as  assistance  in  dealing  with  concrete  IP  problems.  EU 
SMEs value more receiving a reliable upstream of information and advice for implementation than to 
obtain a downstream assistance for concrete IP management.   

  
A comparative analysis of the effectiveness of the three entities of the IHD shows significant 
regional differences. The users surveyed homogeneously value the "raising awareness of the value of 
IP" and "raising awareness of the necessity to secure…" aspects. On all the other aspects listed under 
Figure  6  above,  the  respondents  systematically  rate  the  LAHD  lower  than  the  two  other  regional 
entities.  About  the  objective  to  "Provide  advice  and  support  services  to  EU  SMEs  facing  IPRs 
difficulties” the LAHD appears significantly less effective on the "solving concrete IP problems" and 
"developing  internal  capacities”  aspects.  Around  45%  of  the  survey'  respondents  finds  the  LAHD 
effective whereas the SEAHD reaches more than 70%. For the CNHD, the survey' respondents assess 
the "solving concrete IP problems" aspect in a contrasting way: 58% find the CNHD at least effective, 
whereas 25% find it "ineffective". This is the highest rate of recorded ineffectiveness.  

  
Interviews  provide  some  elements  of  clarification  on  the  level  of  effectiveness  reached  by 
the  LAHD.  According  to  some  interviews,  the  LAHD  would  be  over  academic  and  less  operational. 
One interviewee mentions "…that the material provided for by the LAHD is too basic. I would like to 
have  more  concrete  information  on  IP  protection.  However,  I  am  mostly  explained  that  there  are 
things like patents." This view needs however to be tampered.  When a low number of respondents 
(as  compared  to  the  CNHD  an  SEAHD)  find  the  LAHD  to  be  effective  on  the  "solving  concrete  IP 
problems" and "developing internal capacities” aspects, the number of respondent finding the LAHD 
to be "not effective" is the same as for the two other regions. The LAHD simply totalise more "do not 
know"  responses.  The  gap  found  on  the  effectiveness  of  the  operational  support  provided  by  the 
LAHD is more likely to be linked with a later start (6 months) on a wider and heterogeneous territory 
(19 countries covered against 10 for the SEAHD). The LAHD needs more time to face a wider range of 
specific situations and to improve its visibility across its specific geographical area. 

To conclude, the objective ‘provide supports to EU SMEs’ leads the three regional entities of the 
IHD  to  achieve  important  efforts  to  implement  the  first  line  advisory  services,  to  develop 
publications and IPR materials, to adapt and deliver training sessions and to put in place a relevant 
website. These specific actions have a direct effect on raising awareness for IP management issues 
that  could  explain  the  overall  positive  assessment  in  term  of  effectiveness  from  the  survey' 
respondents  confirmed  by  the  interviews.  Survey  respondents  and  interviewees  agree  that 
helpline,  publication  and  training  services  are  important  and  useful  regardless  of  the  significant 
gaps that can be observed between targets and achievements (see efficiency analysis below). Despite 
these  gaps  and  delays  in  implementing  the  services,  the  IHD  seem  to  have  successfully  provided 
supports services to SMEs facing IP issues in or arising from the target region and have put into place 
a mechanism which enables cater SMEs information and advice needs related to IP protection.  

4.  The survey respondents' views on the IHD effectiveness, by service 

The survey and interviews contribute to give an overview of how users might perceive the benefits of 
the  main  IHD  services  i.e.  helpline,  publication,  website  and  training,  in  terms  of  usefulness  and 
quality.  

 
 
 
o  Effectiveness of the helpline services 

The level of effectiveness of the helpline services varies according to their level of use and the degree 
of usefulness acknowledged by the users.  

The  “deliverables  and  KPIs”  documents  provide  information  on  the  level  of  use.  Data  reported147 
show that the number of enquiries to be answered already exceeds the target set for the IHD (three 
entities included) and this long before the end of the contracting period of the IHD initiative. This is a 
great  achievement.  The  level  of  users  survey  satisfaction  with  response  to  enquiries  is  only 
mentioned  about  LA  (95,7%  achieved  against  99%  targeted). The  corresponding  input  of the other 
regions is unfortunately missing.  

From a qualitative angle, the survey reveals in practice that around 80% of the respondents assess 
the  listed  features  of  the  helpline  services  as  "highly  important"  and  "important".  In  particular, 
providing  "business  oriented  information  in  an  understandable  way  for  non-legal  experts",  and 
offering a "first-line support" in a "user-friendly" way, are the highest ranked aspects (around 90%). 
Asked  about  the  usefulness  of  the  same  features,  nearly  all  survey'  respondents  find  the  helplines 
provided by the IHD to be "useful" or "very useful" (more than 90% at least useful). In China more 
than 60% of the respondents rate these services as "very useful", whereas only 40% of LA ones and 
0% of SEA ones do. These findings are corroborated by interviews of the IHD according to which the 
need among SMEs for the helpline is high. 

  
Similarly,  in  the  views  of most  respondents  to  the  survey, overall,  the  IHD  helpline  service 
sufficiently  covers  the  four  main  IP  topics  (i.e.  IP  Protection,  IP  Checking  of  agreements,  IP 
Management and IP Exploitation). In particular, the coverage of IP protection topics is considered to 
be  sufficient  by  more  than  two  thirds  of  the  respondents.  This  can  be  seen  as  a  positive  result, 
considering  that  IP  protection  is  one  of  the  top  needs  for  advice  and  support  mentioned  by  the 
respondents.  This  indicator  drops  to  50-60%  for  the  other  IP  topics  but  significant  numbers  of 
respondents  were  not  able  to  answer  this  question and  thus  opted  for  the  "do  not  know" option. 
This high percentage of "do not know" answers is not surprising when considering that a significant 
part of survey' respondents considers that the IHD services are not sufficiently promoted. Improved 
promotion of the IHD services might help more users and EU SMEs in general to know about the IHD 
and/or to know well about the scope of services they offer.  

  
Most survey' respondents indicate that an avenue for improvement could be to expand the 
subject scope of the helplines (nearly half of them). This evidence an interest in the development of 
the  helpline  services,  as  well.  In  addition,  neither  the  survey  nor  the  interviews  reveal  any 
dysfunction  or  notable  inadequacy  that  would  need  to  be  fixed.  This  tends  to  confirm  also  a 
significant level of satisfaction with the helpline service.  

To sum up, for the respondents to the survey, IHD helpline services are useful. They cover 

  
aspects and topics that they consider important in relation to their IP management needs.  

o  Effectiveness of the publication services 

147 see below Table 3 in the Efficiency part 

 
 
                                                           
The level of effectiveness of the IHD can also be assessed from the number of publications (guides, 
training  materials,  newsletter,  articles)  written  by  IP  experts  for  SMEs  that  they  offer,  the  level  of 
importance given to them by SMEs and their usefulness. 

  
As for the previous services, a high amount of survey' respondents (around 80%) qualifies the 
different  kind  of  publication  services  offered  (online  library,  factsheets,  FAQs,  events  lists)  as 
"important"  or  "very  important"  (almost  one  third).  In  addition,  overall,  around  90%  of  the 
respondents find the publication services of the IHD to be "very useful" or "useful". However, on a 
closer look, a gap appears between 100% respondents for China on the one hand and 75% for SEA on 
the other hand. Interviews with the IHD confirm that publication services are ranked as an important 
part of the offer, just after the helpline.  

  
Two factors prevent us from appraising in a positive way the level of effectiveness of this 
service. Firstly the "deliverables and KPIs" documents show a limited level of achievement for the 
publications services as compared to the targets set148. It seems that it will be difficult to fil the gap 
for the SEA and LA regions. Secondly, a user satisfaction indicator is missing for this service. 

o  Effectiveness of the online service, website 

  
Granting  access  to  a  useful  and  performing  website  is  also  a  key  feature  of  an  effective 
service  offered  to  SMEs.  The  IHD  have  appropriate  financial  means  to  design  and  maintain  such  a 
website. These means would be considered as well employed when users value high the quality and 
usefulness  of  the  provided  tool.  The  survey  includes  one  question  listing  possible  main  quality 
features of a website like "The website is easily accessible" or "The content is relevant for us" or "the 
website  has  a  clear  layout",  etc.  Height  items  are  listed  for  valuation.  A  high  portion  of  survey' 
respondents  (between  60  and  80%)  agree  that  the  IHD  website  services  display  the  listed  quality 
features. The item "the content is relevant for us" is unanimously agreed to. This is a very good result 
considering  that  it  is  the  most  important  criteria.  Quality  features  like  "the  information  is  easy  to 
find"  or  "the  website  has  a  clear  layout"  are  also  well  rated.  This  outcome  is  however  more 
observable  for  China  and  LA  respondents  surveyed  than  for  the  SEA  ones.  On  the  other  hand, 
ergonomic criteria like search facilities, design, and consistency of navigation throughout website are 
more positively rated in relation to the SEA website than for the China and LA one. Indeed, the last 
two websites gather more negative assessments.   

  
This  overall  good  qualitative  assessment  of  the  IHD  websites  also  appears  from  the 
"deliverables and KPIs" documents. They show a high satisfaction rate for China (4,4/5 against 3,5/5 
targeted  and  for  LA  (80%  satisfied  in  agreement  to  the  target).  There  is  no  evaluation  of  the 
satisfaction of users about the SEA Website.  

  
Overall, the websites, as designed, satisfy the respondent to the survey. They acknowledge 
in  unison  that  the  content  is  clear,  relevant,  and  easily  accessible.  By  contrast  some  ergonomic 
features, which collect more negative assessments, should be analysed to find accurate solutions. 

Beyond  the  qualitative  aspects,  the  level  of  use  must  be  analysed  in  order  to  achieve  the 
performance assessment of the websites. In the "deliverables and KPIs" documents, the level of use 
is  assessed  by  reference  to  the  number  of  visits.  From  reported  data,  it  can  be  derived  that  the 

148 see below Table 3 in Efficiency part 

 
 
                                                           
results are well below the targets149. This is particularly true for the LAHD and SEAHD. The two IHD 
entities  that  tend  to  lag  behind  the  targeted  level  of  use  are  the  ones  which  gather  the  highest 
number of negative responses on ergonomic features like "website easy to find" and "website easily 
accessible".  From  the  interviews  with  the  EC,  it  appears  that  despite  the  fact  that  having  a  single-
entry point for companies via a unique IHD website is a plus, it is a challenge to have a one-size-fits-
all website for the three regional entities150. 

  
  
Although, the websites apparently match the users’ needs as appears from the survey and 
the interviews, it would be recommended to take appropriate measures to improve their visibility 
and accessibility, and by extension their effectiveness. 

o  Effectiveness of the training services 

The IHD provide different kinds of training services on IPR issues to their customers, tailored to the 
needs of SMEs. The effectiveness of the training services depends on some criteria like the different 
training formats offered, the number of completed actions and the number of trainees, as well as the 
level of participants' satisfaction. 

The  survey  provides  interesting  answers  about  the  importance  and  usefulness  of  the  different 
training formats offered by the IHD. The formats cover, for instance, live workshops as well as more 
innovative ones such as online webinars. From the survey it appears that overall more than 60% IHD 
respondents find five of the six listed formats as "very important" or "important". The response rate 
for the "not important" option varies from 5 to 15%. The survey' respondents finds workshops and 
webinars  to  be  equally  important  to  serve  their  needs.  These  two  formats  are  the  ones  which  are 
rated as "most important".  

The survey respondents also find the webinars to be the most useful training service  (more than 
70% of them) but more than 60% of respondents consider also the others mentioned formats to be 
useful. 

Although  the  IHD  entities  have  properly  diversified  the  training  means  to  adapt  to  the  IPR  skills 
development  needs of  SMEs,  the  level of  achievement  appears  more  problematic.  Here  again,  the 
"deliverables  and  KPIs"  documents  reveal  an  important  possible  future  gap  between  targets  and 
achievements151.  The IHD have reached less than 50% of the trainings targets. This is true  for the 
number  of  developed  modules,  as  well  as  for  the  number  of  delivered  sessions.  The  number  of 
participants is proportionally low as compared to the targets. Nevertheless, the reported number of 
participants, in comparison to the number of delivered training actions reveals a relatively important 
level of attendance. This finding could be opportunely validated by providing additional information 
on the average attendance rate e.g. from the attendance sheets.   

The  third element  to  be  considered to  analyse  the  effectiveness  of  the  IHD  training  services  is  the 
valuation  made  by  trainees  based  on  the  evaluation  sheets.  This  data  is  only  provided  by  the  LA 

149 see below Table 3 in Efficiency part 
150 For instance, the LAHD wanted to organise a lot of project management via the websites which was a functionality not 
foreseen by the creators of the central website 
151 see below Table 3 in Efficiency part 

 
 
 
 
 
 
 
                                                           
entity. It reports that 60,6%  of participants  are  highly  satisfied (against  a 95% targeted). The other 
IHD entities do not provide any equivalent information.  

To conclude, the IHD receive grants to implement, among other services, training facilities aimed at 
enhancing SME IPR skills. Various formats (workshop, webinar etc.) were designed and are viewed 
as important and useful by the survey' respondents. However, the number of developed training 
modules, delivered actions, and as a consequence of EU SMEs trained remains limited. This finding 
needs to be taken into account to increase the performance of the IHD in the future. 

c.  Objective 3: Maintain the link with intermediaries and stakeholders  

As already highlighted, the COSME programme mentions that the IHD services will be offered in co-
ordination  with  and  in  direct  response  to  requests  from  European  SMEs  and  their  representatives 
(e.g. sectoral, regional, local associations of SMEs, chambers of commerce)152.  

Checking  how  the  IHD  have  implemented  the  specific  actions  ("provision  of  partnering  services" 
and "joint actions and innovative services to SMEs")  gives a first indication on how the link with 
intermediaries and stakeholders in the EU or in the target country has been maintained. According 
to  the  PO  Assessment  Reports153,  the  link  with  intermediaries  and  stakeholders  including  the 
Enterprise  Europe  Network  (EEN),  industrial  associations,  and  pan-European  business  support 
organisations in the target regions has been maintained during the period154. CNHD and SEAHD have 
attended  a  certain  amount  of  partnering  services  and  matchmaking  events  and  conferences  (e.g. 
until April 2017, the IHD attended 116 events in Asia and in Europe). They did provide participants 
with supporting IPR packs of documents and guides. These actions helped facilitate the strengthening 
of  links  between  EU  SMEs  and  companies  in  the  targeted  area.  In  LA,  this  activity  has  essentially 
taken the form of an initial survey on the training needs of SMEs followed by a few trainings without 
however reporting any participation in matchmaking events. The CNHD, LAHD and SEAHD progress 
reports155  indicate  that  the  three  regional  entities  of  the  IHD  have  also  successfully  developed 
common actions together or with other relevant initiatives (EU SME Centre, EU Gateway projects, 
EEN, ELAN network…) both in Europe and in the targeted areas. This collaboration has been fruitful 
with  development  of  joint  trainings  and  dissemination  activities,  mutual  web  linking,  joint  content 
materials  and  joint  participations  in  events  and  trainings.  However,  the  development  of  a  joint 
website  was  not  successful.  The  efficiency  analysis  of  the  objective  to  maintain  the  link  with 
intermediaries  and  stakeholders  will  however  reveal  discrepancies  between  the  regional 
performances.  

152 See ANNEX to the Commission Implementing Decision concerning the adoption of the work programme for 2014 and 
the  financing  for  the  implementation  of  Programme  for  the  Competitiveness  of  Enterprises  and  small  and  medium-sized 
enterprises 
153 Non publishable internal source 
154 The China 3/11/2016, LA 16/12/2016 and SEA PO Assessment Reports mentions that the project has put emphasis on 
cooperation (dialogue, joint coordination like implementing an annual cooperation action plan, signature of MoU) with  the 
most  relevant  EU-funded  projects  notably  the  EU  SME  Centre  and  EU  Gateway  projects  (in  China);  ICI+  projects,  EU 
Business Avenues, EU-ASEAN Business Networks (in SEA); ELAN Network, ELAN Biz and AL INVEST (in LA) and the 
European IPR SME Helpdesk resulting in joint dissemination, joint participation in events and trainings (onsite or in Europe) 
or  webinars,  by  exchanging  internally  best  practice,  sharing  joint  actions  or  developing  an  international  'train-the-trainer' 
programme for intermediaries. 
155 Non publishable internal source 

 
 
 
 
                                                           
On  the  qualitative  aspect,  the  above  Figure  7  highlights  that  just  over  50%  of  the  survey' 
respondents assesses as effective the capacity of the IHD in "building long term partnership with 
relevant EU networks". Additionally, only a bit more than 40% of the respondents assesses the IHD 
to  be  effective  in  "building  networks  and  relations  to  experts  and  institutions  in  the  relevant 
countries".  This  last  finding  is somewhat  worrying  since  the  specific objective involves  the  IHD to 
help EU SMEs work well together with local authorities and service providers in IP. On a closer look, 
in comparison to the other users in the sample, in LA only a very low number of respondents (30%) 
found the IHD to be effective in building networks and relation to experts, and long-term partnership 
with  relevant  EU  networks.  This  finding  could  be  linked  to  the  fact  that LAHD does  not  report  any 
participation to partnering services and matchmaking events.   

According to the interviews and in line with the survey, the IHD could become more effective by 
increasing cooperation with other EU activities, as well as with national activities. Around 85% of 
the IHD survey' respondents rate highly the importance of the extent of the network managed by the 
IHD and that of having local support in the region. 

To  conclude,  while  this  objective  of  the  IHD  service  is  rated  highly  by  the  respondents  to  the 
survey,  the IHD  are  seen only  effective  to  a certain  extent  in  building  long  term  partnership  with 
relevant EU/other stakeholders.  

d.  Objective 4: Adapt the work to the target group on demand  

Overall, during the period, the IHD have tried to answer the requirement for improving the quality 
of  their  services  to  the  group  of  users  on  demand156.  Publication  services  provide  significant 
examples of such implementation with efforts in using a jargon-free language understandable by 
SMEs  having  no  or  only  basic  knowledge  of  IPRs,  or  translating  where  demands  exist.  Overall,  the 
quality  of  the  language  has  improved  as  compared  to  the  previous edition of the  IHD.  The  project 
focussed  also  in  developing  new  publications  centred  on  particular  economic  sectors  relevant  to 
the  geographical  area  at  issue  (e.g.  guide  on  IP  considerations  for  Automotive  Industry  in  SEA  or 
guide  to  IPR  protection  in  China  for  the  Fashion  and  Design  Industry)  or  about  country(ies)  not 
previously geographically covered (e.g. factsheets and FAQs related to Mexico, Colombia, Peru and 
Costa Rica, or IP factsheet on Macao). The  sectorial approach and extension to new countries has 
also been followed in the training activities offered (e.g. nano and biotechnologies sector-oriented 
modules developed in LA or development of webinars capturing a wide diversity of sector in China). 
To  be  mentioned  as  well:  in  LA  the  partnering  activity  started  with  a  survey  to  understand  the 
training  needs  of  SMEs,  after  which  a  few  trainings  were  successfully  developed.  These  examples 
show that the IHD pay specific attention to adapt their activity to the group of SMEs on demand. The 
outcome is that the current action of the IHD is not a mere continuation of the services previously 
provided:  it  has  extended  in  scope  by  covering  both  specific  economic  sectors  and  new 
geographical areas.  

Evidence tend to show that the respondents to the survey and interviewee value the customization 
aspect  of  the  current  IHD  services  which  varies  over  time.  Each  time  the  survey'  respondents  are 
asked about the relevance and usefulness of the services offered, the responses are positive in a high 
proportion. The publication services in general are perceived as useful by the respondents. 90% find 

156 Non publishable internal source 

 
 
 
                                                           
them useful or very useful, while 45% of the respondents state that they use these services at least 
occasionally (against only 25% for the helpline). Eventually, the interviews of the IHD reveal that one 
factor  that  makes  the  IHD  particularly  relevant  for  businesses  is  that  the  IHD  services  are 
customised (by key sector, MS, target country) and that the IHD provide their services accordingly. 
Interviews from the EC services confirm this trend as IHD are seen there as having a general ability to 
communicate in an SME like language and to adapt easily to new partners/environments. 

In addition to publication service, the IHD are asked to offer services like trainings or advice (through 
the  helpline),  customized  to  the  need  of  their  users.  Between  52  and  55  %  of  the  survey' 
respondents rate  as effective the  IHD capacity to solve concrete IP problems in the third countries 
targeted and in developing internal capacities in SMEs to deal with IP matters. 

The IHD took care to tailor their work to the target group on demand. The external feedback shows 
that it remains adapted to the work of this target group to some extent (publication services and 
provision of first line information). However, the overall perception of the entire IHD offer seems 
more  nuanced.  As  already  highlighted,  the  capacity  of  the  IHD  to  deal  with  specific  IP  problems 
faced by their users and to help them develop internal capacities to deal with IP protection could 
apparently  be  improved.  On  these  aspects  (support  to  IP  problem  solving  and  development  of 
specific IP skills) coherence needed with the services that law firms and consultancies offer, implies 
some limitations (see coherence part).   

Further analysis on regional variations will be provided in the efficiency part of the report. 

Effectiveness of the IHD measure in achieving the three specific objectives 

e. 
concerned markets 

Specific  objective  1:  reduce  the  risk  of  counterfeit  products  entering  the  EU  market  from  the 

implemented  by  the 

The  specific  actions 
(helpline,  publication,  trainings,  website, 
communication  and  dissemination)  are  aimed  at  enabling  the  EU  SMEs  to  control  their  IPR 
management  and  defend  their  intellectual  assets  before  entering  and  while  doing  business  in  the 
target regions.  

IHD 

The following chart157 on IPR Infringements shows in particular a slight increase in the high number of 
articles suspected of violating IPRs (while  the number of cases opened by the EU customs remains 
stable). 

157This  chart  appears  on  the  EC  webpage  dedicated  to  "IPR  Infringements:  Facts  and  figures"  which  publishes  the  2016 
Report on EU customs enforcement of IPR at the EU border  

 
 
 
 
                                                           
6. 

Number of registered cases and articles

Source: European Commission: IPR Infringements: Facts and figures 2016 - 

6. 
https://ec.europa.eu/taxation_customs/business/customs-controls/counterfeit-piracy-other-ipr-violations/ipr-
infringements-facts-figures_en  

The  2016  Report  on  EU  customs  enforcement  of  IPR  at  the  EU  border158,  for  its  part,  stresses  in 
particular that 
- China, Hong-Kong, Singapore, Thailand, Vietnam and Malaysia are among the countries of origin,  
- China continues to be the main source country from where IPR infringing goods are shipped to the 
EU (except for certain goods like alcoholic beverages). 

Furthermore, according to the 2017 Situation Report on Counterfeiting and Piracy in the European 
Union159 prepared by the EUIPO and Europol, "seizures at the EU external border are estimated to be 
only a small part of the total amount of imported counterfeit goods". Such goods are otherwise sold 
online, subsequently distributed via parcel and postal services and difficult to seize as "targeting and 
controlling suspicious shipments in postal freight are a great challenge".  

In front of such evidences no conclusion can be derived in relation to the link with IHD intervention 
because of its inner limited scope.   

However, based on several observable factors, it is possible to conclude that the IHD initiative might 
contribute to this result. By helping EU SMEs to be increasingly aware of the value of IPR, and to 
adopt adequate protection behaviours and methodology,  the IHD, to some extent, contribute to 
reducing  risks  of  counterfeit  products  entering  the  market.  Evidence  shows  for  instance  that 
around half of the sample of SMEs surveyed having experienced the IHD services express that they 
start to "take steps to manage IP” or that they "consistently now manage IP in or outside the EU”.  

158 See previous note 
159 2017 Situation Report on Counterfeiting and Piracy in the European Union 

 
 
 
 
  
                                                           
During  the  stakeholder's  consultation,  28  companies  reported  that  –  before  the  contact  with  the 
(IHD) – they were "only just aware that IP is an issue". Out of these 28 companies, 12 reported that 
after the contact with the (IHD) they were beginning to take steps to manage IP inside and outside 
the EU. Therefore, the survey evidence seems to indicate that almost half of the respondents (43%) 
reported  an  improvement  in  their  ‘IP  management  status’  after  the  contact  with  the  IHD.  For 
respondents citing a higher level of IP management skills for the situation before the contact with the 
HD, these improvement rates are similar (around 40%).”  

  
Despite the low number of responses, it seems that the action of the IHD has a positive effect 
in  developing  the  IP  management  skills  among  users  of  the  IHD.  This  result  should  obviously  help 
SME’s users to come to have a better control on the risk of infringement. 

However, as previously analysed, in a territory like China where infringement is a huge issue, the IHD 
reaches  only  3%  of  EU  SMEs  exporting.  The  more  this  percentage  increases,  the  more  impact  on 
reduction of risks of counterfeiting products entering the EU market will be observable. 

a. 
concerned markets  

Specific objective 2: Protect SME's valuable IPR before entering and while doing business in the 

As highlighted above, the specific actions of the IHD should generate a positive effect in developing 
SMEs  IP  management  skills  to  reduce  the  counterfeit  risk  and  to  put  their  IPR  protection  under 
control. Evidence shows an interest in users getting support in IP protection and registration both 
before  and  after  doing  business  in  the  targeted  areas.  For  instance,  a  large  majority  (80%)  of 
survey' respondents agree that  the assistance provided by the IHD is essential when planning to 
expand to a third country as well as that it is important when having entered a third country. As 
already  seen,  the  respondents  to  the  survey  need  advice  and  support  mostly  in  relation  to  IP 
protection  (59%)  and  registration  (41%)160.  From  a  purely  qualitative  angle,  several  case  studies 
presented by IHD in their Impact Assessment run in 2016 show examples of how the IHD helped EU 
SMEs to protect their IPR before entering the target area.  

As a matter of facts, the IHD have put into place means to protect IP and to register IPR both before 
entering the targeted area and after EU SMEs did. The presence of local support, as well as presence 
in  the  EU  is  key  to  maintain  services  continuity  both  in  time  and  space.  At  first  sight,  from  the 
evidence  collected,  there  is  no  feedback  showing  that  EU  SMEs  would  benefit  from  better  IP 
support  services  before  than  after  entering  the  targeted  area.  The  opposite  is  also  true.  Some 
interviews imply however that the LAHD meets difficulties in providing local support to SMEs with 
the  same  degree  of  effectiveness  as  in  Europe.  Some  justification  like  a  "too  academic  approach", 
"insufficient country specific publications" as well as the location of the experts’ offices in Europe and 
not in LA, are put forward to explain these difficulties. 

b. 
service providers to enforce their IPR protection 

Specific  objective  3:  Help  SMEs  to  find  and  constructively  work  with  local  administrations  or 

160 see above Figure 7 

 
 
 
 
 
 
                                                           
While more than 80% survey' respondents find that the assistance that the IHD provide is essential 
once  they  have  entered  the  third  country,  and  the  IHD  members  themselves  consider  them  as 
directly  impacting  on  increasing  the  ability  of  EU  SMEs  to  collaborate  with  local  authorities  and 
service  providers for enforcement  purposes161, the survey shows on the other hand that  only a  bit 
more than 40% of the respondents consider that the IHD are effective as a mechanism to building 
networks and relations to experts and institutions in the targeted countries covered. The interview 
with  the  IHD  and  the  EC  services  corroborate  this  finding  by  showing  that  awareness  of  the  IHD 
services in Europe is limited.  

This  could  indicate  only  a  moderate  level  of  effectiveness  of  the  IHD  in  reaching  the  effect  of 
helping EU SMEs to find and constructively work with local administrations or service providers to 
enforce their IPR protection in the views of the respondents to the survey.  

6.1.2.  Where  expectations  have  not  been  met,  what  factors  have  hindered  their 

achievement?  

Focusing  on  the  specific  actions  (helpline,  trainings, publications,  website  and  partnering  services), 
the effectiveness analysis shows that only one service offered by the IHD i.e. the helpline fulfils the 
targets  set.  The  IHD  face  problems  in  providing  the  remaining  services  in  a  full  satisfactory  way. 
External or internal hindering factors explain them.  

A  first  hindering  factor  occurred  in  SEA.  The  IHD  was  not  operative  in  SEA  until  October  2015 
(instead  of  January  2015).  This  delay  was  caused  primarily  by  changes  in  the  constitution  of  the 
consortium which occurred during grant preparation. In addition, it is only as from December 2015 
(when the new IP advisor became part of the EuroCham Vietnam payroll) that publications, trainings, 
policy feedback and IP support during matchmaking events services started to gain momentum. This 
explains that for these services, the consortium is still lagging behind the targets162.  

Another external hindering factor lies with difficulties in LA in reaching out to Brazilian and Mexican 
stakeholders  essentially  due  to  a  lack  of  cooperation  of  stakeholders  who  perceive  the  IHD  as  a 
competitor. This factor impacted negatively the level of achievement of trainings163.  

Among the internal hindering factors, the IHD initiative  has not  been implemented satisfactorily in 
relation to the development of a joint website164. This task has been severely delayed (completed in 
March 2016 i.e. almost 12 months after the expected date). In addition, the quality level of the joint 
portal  did  not  meet  the  expectations  of  the  IHD.  Several  functionalities  were  missing  resulting  in 
various dysfunctions and extra work (manual registrations of training participants after an event, or 
need to use a previous blog in LA). Eventually, the procedure for addressing the issues/bugs was not 
quick and effective enough. The delay in making the website available and following implementation 
problems  might  have  partially  hindered  the  achievement  of  certain  services  e.g.  website  and 
trainings.  

Another  concern  in  LA  lies  with  a  lack  of  attendance  of  the  IHD  to  matchmaking  events.  This 
hindering  factor might  have  restricted  the  possibility  for  EU  SMEs  trading  (or  planning  to  trade)  in 

161 see Study, interview synthesis 
162 see Project Officer Report in SEA 
163 see Project Officer Report in LA 
164 see Project Officer Report in each targeted region 

 
 
 
                                                           
this area to match (e.g. when EU SMEs would refrain from participating because they do not benefit 
from  support  in  relation  to  IP  protection)  or  match  in  a  safe  way  (e.g.  when  they  would  not  have 
taken care to IP protect their assets before negotiating with trade partners because they would not 
have been made aware of the need to protect them due to the lack of attendance of the IHD) trading 
partners. 

Conclusion on effectiveness 

The  IHD  are  expected  to  serve  the  COSME  programme  general  objectives  of  strengthening 
competitiveness and promoting growth and specifically improving access to markets at global level. 
They should thus help SMEs to enter and grow on the Chinese, SEA, and LA markets, while protecting 
their  IP  assets,  preventing  them  from  risk  of  counterfeit,  and  building  constructive  relations  with 
local administrations and services providers.  

From  the  evidence  reviewed,  and  with  the  usual  caveat  as  regards 
representativeness of the sample analysed, the following trends appear:  

in  particular  the 

  EU  SMEs  would  particularly  value  getting  advice  and  support  on  the  different  format  of  IP 

protection as well as information on registration and other procedures.  

  The  IHD  would  be  perceived  as  specifically  effective  in  increasing  awareness  for  IP.  The  tools 
implemented  to  that  end,  like  the  helpline,  trainings  and  publications  effectively  would 
contribute to cater the EU SMEs need for advice and reliable information both before and after 
entering the targeted third countries markets. 

  A  significant  part  among  users  of  the  IHD  report  having  improved  their  IP  management  after 

having been in touch with the IHD.  

  The  level  of  effectiveness  appears  to  be  lower  as  regards  solving  concrete  IP  problems  in  the 
countries covered by the IHD and developing internal capacities to deal with IP issues. This last 
finding  appears  however  acceptable,  considering  first  that  the  IHD  rightly  prioritised  the 
provision  of  information  on  IP  (helpline,  publications,  trainings).  Second,  the  range  of  services 
offered by the IHD is not supposed to overlap with the services provided by the private sector. It 
will  always  be  necessary  to  have  in  mind  the  border  between  providing  first-line  support  and 
raising IP awareness on one hand, and delivering legal support services on the other hand.   

  An  element  that  the  IHD  should  take  care  of  is  to  develop  network  with  local  experts  both  in 
Europe  and  in  the  targeted  countries.  Results  between  the  three  targeted  regions  are  very 
heterogeneous. The LAHD underperforms as compared to the two other regional entities of the 
IHD.  Improving  the  performance  of  the  IHD  in  networking  development  could  in  particular 
involve increasing significantly their promotion at local level. 

Although the delay in implementing the IHD services in SEA, as well as the lack of a joined website for 
one  year  impacted  the  figures  (e.g.  First  six  months  in  SEA  and  absence  of  on  line  registration  of 
users  in the entire IHD), positive  factors could lead to assume that, on the medium/long term, the 
IHD could fulfil the objectives of the initiative. Basic instruments have been put into place to reach 
SMEs  and  enable  them  efficiently  face  IPR  issues  before  and  after  entering  the  targeted  third 
countries markets. These instruments are an effective helpline service, trainings, useful publications 
and a website. Key success factors of the IHD on the long term lies on their capacity to first improve 

 
 
 
 
the quality of these instruments already largely valued by the users, second, tackle the problems of 
insufficient promotion of their activities, and third improve partnering services. 

6.2.  EFFICIENCY  

6.2.1.  What  are  the  costs  incurred  to  the  IHD,  the  public  authorities,  and  the 
other  stakeholders  to  implement  the  initiative  (as  defined  by  the 
Regulation, and follow up grant agreements)? 

The  IHD  project  is  funded  by  the  European  Union  via  the  COSME  programme  established  by  the 
COSME regulation (EU) No 1287/2013. The budget involved is Euro 7 200 000 (Euro 2 400 000 for the 
CNHD, Euro 2 600 000 for the SEAHD, Euro 2 000 000 for the LAHD), plus an additional contribution 
of Euro 200 000 to develop and maintain the common portal. The EU grant is limited to a maximum 
co-funding rate  of 90%  of eligible costs. As already  noted, the contracts are for 36 months (CHINA 
and SEA) or 30 months (LA). 

The COSME programme requires from the IHD periodical and detailed reports (quarterly, annual, and 
biannual  interim  reports)  of  their  activity  reviewed  through  7  KPIs  that  match  with  the  specific 
actions.  

The achievement of these objectives implies the use by the IHD of their European funding, to provide 
SMEs with the following four main services:  

  Helpline services to answer enquiries and give confidential advice  
 

Publication  services  (training  materials,  guides,  articles,  newsletter  content  or  other 
publications) 

  Online (website) services 
 

Training services 

The efficiency analysis aims at checking how optimal the use of the allocated financial resources by 
the IHD is, in view of the achievements.  

The table below shows how the IHD split their costs among various expenditure items. 

Table 2:  Costs share among various services 

Grants 
(euros) 

Direct 
personnel 
costs (euros) 

Direct costs of 
subcontracting 
(euros) 

Helpline  Trainings 

Publications 
& materials 

Overhead costs 
(dissemination, 
website, 
partnering…) 

 CHINA 
SEA 
LA 
Source: IPR Helpdesks 

2.400.000  2.154.217 
2.600.000  1.984.590 
2000.000  1.346.243 

308.426 
421.250 
235.345 

48% 
31% 
31% 

19% 
33% 
23% 

14% 
17% 
10% 

19% 
20% 
36% 

It shows that there is not a uniform allocation of expenses among the three IHD regional entities. In 
China almost a half of the cost  (48%) relates to the helpline  service. This  benefits the CNHD which 
reaches higher amounts of registred helpline users and handles more enquiries than the SEAHD or 
LAHD do (see Table 3). The SEAHD evidences a more balanced cost allocation between helpline (31%) 
and training (33%) services. The SEAHD reaches however lower results than the CNHD for these two 

 
 
 
 
 
services. More than two-thirds of the cost of the LAHD are dedicated to helpline service (31%) and 
the "overheads costs" (36%). The LAHD reaches a satisfactory amount of registred helpline users but 
less important ones in terms of website use and partnering service, despite significant expenditures 
incurred.  For  the  three  regions,  publications  services  are  the  poor  relative  expenditure  item.  This 
could explain why results  in this area are well below  the targets (see  Table 3). Two cost  allocation 
strategies  can  therefore  be  distinguished.  The  CNHD  one  which  favours  the  helpline  service,  that 
becomes key actor to the overall performance of the HD. Partnering and dissemination efforts help 
the  CNHD  attract  a  significant  amount  of  users  and  handle  a  large  number  of  enquiries.  Next,  the 
useful  helpline  generates  a  positive  effect  on  trainings.  Publications  and  website  maintain  the 
relationship with the IHD users. In SEA and LA, cost allocation strategy is distinct. The helpline service 
does not play a pivotal role as important as for the CNHD. KPIs from the other services benefit less 
from the good helpline results. In terms of efficiency, results show that the cost allocation strategy in 
CHINA  seems more  rewarding.  However,  analysis of  KPIs  and,  above  all, of  average  costs  achieved 
masks more contrasted patterns. 

6.2.2. 

To what extent are the costs justified and proportionate, given the benefits 
achieved by the IHD?  

To ensure a proper monitoring and evaluation of the specific measure, each regional entity of the 
IHD was asked to measure and record the following essential performance indicators.    

• Number of inquiries from EU SMEs answered and SME satisfaction and feedback on quality;  
• Number and quality of training and awareness events, including virtual events organised as well as 
number of SMEs and business support organisation participants 
• Number of visits and hits on the website and SME user satisfaction of the website;  
• Number and quality of guides, factsheets, e-learning modules and other publications targeted at 
SMEs and business support organisations;  
• Improved performance of end-users in relation to targeted issue;  
• Clients performance assessment (growth of sales, jobs created/maintained, new products or 
services developed).  

Other  performance  indicators  were  detailed  in  the  framework  of  the  requested  quarterly,  interim 
and  final  reports.  The  level  of  achievement  of  the  above  mentioned  KPIs  is  a  significant  way  to 
analyse whether costs are justified and proportionate.  

A comparative analysis of the most significant targeted/achieved results of the specific actions of the 
IHD was performed on the basis of the three "deliverables and KPIs" documents (one for each IHD 
region) covering the reporting period. These results appear in the following table:  

Table 3: Most significant targeted/achieved results of the specific actions of the IHD  

CHINA 

SEA 

LA 

Reporting period 
covered (months) 
GRANT (euros) 

28 (out of 36) 

28 (out of 36) 

22 (out of 30) 

2.400.000 

2.600.000 

2.000.000 

Targeted  Achieved  Targeted  Achieved 

Targeted 

Achieved 

 
 
 
 
 
 
  
Numbers of enquiries 
Number of helpline 
users 
Users satisfaction with 
response  

Training sessions to be 
delivered 
Webinars to be 
delivered 
Participants 

Quality assessment 

WP1 FIRSTLINE ADVISORY SERVICES 

1500 

1509 

550 

652165  

800-1200 

687 

1000 

1347 

1000 

590 

600-800 

1217 

3.75/4 

N/A 

3.75/4 

N/A 

99% 

95,70% 

WP3 TRAININGS 

260 

125 

260 

60 

37 

72 

60 

38 

122+107 

36 

60 

6 

15000 

7682 

15000 

3578 

N/A 

N/A 

N/A 

N/A 

3660 
95% highly 
satisfied 

1655 
60,6% 
good 

 WP5 PARTNERING SERVICES 

Number of events 
attended 

30 

78 (4 in 
EU) 

60 

38 

30 

0 

WP6 WEBSITE 

Number of visits  

240.000° 

119.692 

125.000° 

32.479 

180.000 
(estimated) 

20.000 
(estimated) 

User satisfaction  

3,5/5 

4,4/5 

3,75 
WP2 PUBLICATIONS 

N/A 

80-100%  

80%  

New guides & other 
publications 
Case studies 
Articles published 
Trainings modules 
developed 

Nb of subscribers to 
Newsletter 

Number of SME 
reporting taking follow-
up actions 
Number of customers 
reporting positive 
development of their 
business situation 
Source: IPR Helpdesks 

35 

12 
450 

35 

20 

8 
239 

20 

60 

15 
450 

39 

32 

9 
230 

11 

25 

30 
236 

N/A 

10 

9 
211 

25 

WP4 MONITORING OF IP POLICIES & LEGISLATION 

5.000 

N/A 

5.000 

975 

N/A 

N/A 

OVERALL CLIENT SATISFACTION ASSESSMENT  

60% 

N/A 

60% 

N/A 

1000 

N/A 

80% 

N/A 

70% 

N/A 

400 

N/A 

This table presents a selection of most relevant KPIs. It gives an overall vision of the achievements of 
the IHD in relation to the resources granted.  

165 770 in June 2017 as reported in the Progress Report of the SEAHD 

 
 
  
  
  
  
 
 
 
 
 
 
 
  
  
 
 
                                                           
Regarding the assessment of progress toward the objectives, the figures show that globally the IHD 
strive to achieve the specific actions with varying degree of success to fulfil their KPIs. 

The CNHD appears as the most efficient IHD. It over performs in relation to the number of handled 
it  made  significant  promotion  efforts  favouring 
inquiries  and  registered  users.  Similarly, 
dissemination  of  IP  content  in  particular  thanks  to  an  active  participation  to  events  (78  events 
attended against 30 targeted). As regards other KPIs, the level of achievement is below the targeted 
objectives. However, these results are not worrying. They could imply that the CNHD is on an upward 
trajectory,  which  would  probably  lead  to  achieving  closer  to  the  targets  by  the  end  of  the  period 
considered.  

The SEAHD ranks as a second with a medium level of efficiency with significant gaps between targets 
and achievements. These backlogs are  partly due to a belated start166, which hampered its  pace of 
development.  Nevertheless,  the  level  of  achievement  on  the  different  KPIs  are  less  worrying 
considering that the SEAHD reached a normal cruising speed as from the second year only.  Now it is 
clear that the helpline works well, the website is in place, trainings are delivered and the publications 
are  well  on  track.  The  SEAHD  is  also  concerned  with  promotion  as  evidenced  by  their  events 
attendance  efforts.  The  SEAHD  experiences  more  significant  gaps  with  the  targets  than  the  CNHD, 
but seems to follow a good development rhythm.  

The LAHD performs below the targets on almost all KPIs. Fortunately, the LAHD scores very well on 
the registered number of helpline users (1217 out of 800 targeted) with a very high satisfaction rate. 
On  the  first  line  advisory  service,  the  LAHD  performs  better  than  the  SEAHD,  considering  that  the 
LAHD  has  achieved  a  similar  number  of  enquiries  in  six  months  less  of  operation.  As  regards  the 
other  specific  actions,  underperformances  are  clearly  perceptible.  On  trainings  for  instance,  the 
LAHD mentions 25 trainings modules developed, leading to 60 training sessions delivered, gathering 
1655  participants  (27  per  session),  while  the  CNHD  with  20  modules  developed,  trained  7682 
participants  in  125  sessions  (61  per  session).    While  the  SEAHD  and  the  LAHD  have  delivered  the 
same amount of training session, the SEAHD has a number of participants twice as important (3578 
against 1655). A critical issue for the LAHD is the lack of participation in business events organised by 
third  parties  (0  events  attended)  from  which  the  other  IHD  regional  entities  get  most  of  their 
enquiries.  It  is  easier  for  the  CNHD  than  for  the  SEAHD  and  LAHD  to  achieve  the  specific  action 
"provision of partnering services". This has to do, to some  extent,  with the higher interest  that EU 
companies  have  in  the  China  market.  The  LAHD  and  SEAHD  encounter  difficulties  when  trying  to 
convince stakeholders to organise meetings targeting these two regions. China is an easier market. 
On  this  basis,  a  lower  performance  from  the  LAHD  might  be  understood  (like  that  of  the  SEAHD 
which shows 38 events attended against 60 targeted) but not a complete lack of participation. The 
LAHD should be more pro-active in engaging in third-party events. The very low amount of visits on 
the  website  is  another  issue  for  the  LAHD.  This  can  be  related  to  the  insufficient  and  delayed 
development  of  the  website  by  the  external  responsible  entity.  As  a  result,  the  LAHD  blog  was 
available only one year after the start of the project. In light of these results, it is difficult to conclude 
for the LAHD on an onward trajectory which could lead to catching up on the backlog by the end of 
the period considered.  

166 Due to unexpected changes in the composition of the consortium, see on page 30 

 
 
                                                           
To assess cost/benefit ratio, comparing data on Tables 2 and 3 allow determine the average cost per 
unit  of  various  services.  This  gives  a  deeper  insight  into  the  level  of  efficiency  required  from  each 
regional entity of the IHD and reached by them. 

Table 4: Average costs (euros) of the services by work unit  

Average cost of one registered 
helpline user 
Average cost of one handled 
enquiry  
Average cost of one trainee 
Average cost of one training 
session 
Average cost of one piece of 
publication 
Average cost of one website visit 

CHINA 

SEA 

LA 

Targeted  Achieved  Targeted  Achieved  Targeted  Achieved 

1034 

597 

615 

693 

596 

251 

689 

31 

533 

1119 

47 

53 

867 

173 

417 

99 

445 

161 

1462 

2246 

2391 

6301 

2302 

4042 

612 

838 

685 

985 

595 

1,26 

1,96 

2,22 

6,65 

1,05 

491 

6,91 

This table evidences first that the overall efficiency of the IHD is not very high. The targets appear as 
not  properly  defined  considering  the  distinct  cost  allocation  schemes.  For  instance,  as  regards  the 
helpline,  by  dividing  the  targeted  KPIs  by  the  costs  to  reach  them,  the  average  targeted  cost  for 
handling one enquiry can reach until euro 1119! Similarly, by devoting 48% of its costs to the helpline 
service, the CNHD reaches an average target cost of euro 1034 for each registered user. Fortunately, 
by largely exceeding their target, the CNHD limits this cost to euro 597 (over the 28 months reporting 
period). However, the LAHD performs apparently better in this area. By assigning 31% of its budget 
to the helpline service, the LAHD reaches almost the same amount of users in 22 months than the 
CNHD does in 28 months. This way, the LAHD limits the average cost of one user to euro 251. The 
LAHD appears as the most efficient, in relation to the average cost for one enquiry, by reaching euro 
445  (against  euro  533  and  867  for  respectively  China  and  SEA).  In  contrast,  the  CNHD  reaches  an 
average cost for training and website use which outperforms the two other regions. This result from 
the  CNHD  allocating  a  higher  portion  of  their  budget  to  the  helpline  service167  and  to  the  positive 
impact  generated  on  the  other  services.  Question  arises  as  to  why  the  LAHD,  which  achieves  a 
number  of  helpline  users  that  is  almost  as  important  as  the  one  that  the  CNHD  reaches,  does  not 
benefit  from  an  identical  knock-on  effect  on  other  services.  One  explanation  could  be  that,  while 
having a more efficient helpline, the LAHD deals with a relatively low number of enquiries (less than 
half  of  the  CNHD  one).  The  knock-on  effect  on  trainings  or  website  use  can  therefore  be  limited. 
Similarly  the  SEAHD  which  has  the  most  expansive  and  less  developed  helpline,  benefits  only  to  a 
very limited extent from this knock-on effect. The CNHD, for its part, largely compensates a relative 
loss  of  efficiency,  due  to  a  high  level  of  expenses  being  focussed  on  the  helpline  services  (48%  of 
cost)  with  significant  efficiency  gains  on  the  training  and  website  services.  As  a  matter  of  fact,  a 
training  participant  costs  euro  47  in  China,  against  euro  173  and  161  in  SEA  and  LA,  respectively. 
Similarly, the CNHD performs one training session for a cost of euro 2246, while the equivalent cost is 
euro 6301 for the SEAHD and euro 4042 for the LAHD. Therefore the cost allocation in China appears 
as globally more efficient. The publications item shares the lowest cost expenditure in the three IHD 

167 See Table 2 

 
 
 
 
 
                                                           
regions (ranking from 10 to 17%). None of the three IHD entities reaches the targeted KPIs. This could 
mean  that  the  IHD  entities  do  not  devote  sufficient  time  and  means  to  this  service.  Actually,  they 
have to follow numerous KPIs in relation to the publication services. Preparing and publishing all the 
articles,  guides,  video/audio  podcasts,  newletters,  FAQs,  infographics,  case  studies...  that  they  are 
asked for, would force them to dedicate an important portion of their resources to the detriment of 
other services. The IHD have not abandoned this specific action. They have clearly not prioritised this 
action to the extent required by the KPIs. The LAHD displays the lowest unitary cost (euro 491) but 
limits  the  amount  of  time-and  resource-consuming  publications  (news  guides,  video  podcasts, 
specific materials…). In China and SEA, the time devoted to publications compared to the number of 
items  produced  does  not  reveal  a  very  efficient  result.  However,  without  reaching  the  numerous 
amounts  of  KPIs  for  the  various  publications  targeted,  the  IHD  offers  to  their  users  an  acceptable 
amount of information that the user survey finds important and useful. It is likely that the IHD cannot 
dedicate  more  than  a  minimum  threshold  of  cost  to  publications.  Their  level  of  efficiency  might 
deteriorate  even  more  considering  that  time  for  preparing  certain  publication  items  might  appear 
disproportionate in light of the expected level of consultation on the website.  

The efficiency analysis shows that the IHD are struggling to optimise the cost/benefit ratio within the 
framework of the KPIs, a system which forces them to arbitrate between various targets that are not 
always appropriate considering potential use by SMEs. Some targets are not sufficiently ambitious. In 
particular,  it  is  more  than  likely  that  the  three  regional  entities  of  the  IHD  will  reach  a  number  of 
registered users and handled enquiries significantly higher than the targets set. The average cost will 
decrease accordingly. Other targets appear, to the opposite, largely oversized (KPIs for publications, 
amount of website visits) and require huge resources to implement them even only partially.  

While  the  average  unit/cost  analysis  evidences  that  the  cost  allocation  is  more  efficient  in  China, 
where the helpline plays a pivotal service paper, it would be excessive to devalue the efficiency level 
of the LAHD or the  SEAHD as compared to that of the  CNHD. The SEAHD and LAHD cover various, 
heterogeneous countries with different jurisdictions requiring also distinct expertise. In this context, 
the two regional entities need to gain more  experience  to set  appropriate organisation, processes, 
tools  and  networks.  To  improve  the  costs/benefits  ratios,  experience  shows  now,  that  it  will  be 
necessary  to  focus  on  the  most  appropriate  tasks  that  provide  the  best  level  of  effectiveness  in 
relation to the keys objectives. The authors of the external study underline that "inefficiencies arise 
in relation to some sub-services, where costs might be higher than possible benefits. This relates, for 
the China helpdesks, for example to the development of e-learning modules. Developing e-learning 
modules is costly and the respective additional benefits are hard to measure. For the SEA helpdesk, 
partner  bulletins  –  where  the  helpdesk  tries  to  insert  information  into  the  publications  and  info 
channels of partner organisation – were also a difficult terrain. Many partner organisations were not 
particularly open to that kind of activity, and it proved difficult to sync the timing and scope of the 
respectively to-be-inserted information between the HDs and the partner organisations involved. The 
issue  of  the  website  of  the  LAHD  clearly  impacted  also  efficiency  of  (web-site)  service  provision 
negatively. And it is also clear that serious games, and to a lesser extent the podcasts, have proven 
not only largely ineffective but also inefficient activities. We deduct this from the very low level of 
demand, as evidenced by the respective KPIs". 

To conclude, the assessment of the results achieved in light of the KPIs show that during the period 
considered,  the  IHD  experimented  difficult  implementation  phases  which  led  to  more  or  less 

 
 
delays. The CNHD has apparently found an organisation and a strategy that should help it to reach 
the expected targets with better  efficiency than both others. The SEAHD and LAHD are seriously 
involved in the implementation of some specific actions i.e. helpline,  publications,  and trainings. 
The one-year under performing website hindered their communication and dissemination actions.  
The SEAHD and LAHD should seek ways to participate more to business events. The "provision of 
partnering service" is one of the specific actions that the IHD have to implement. Although, on the 
basis of the evidence gathered, this action seems more difficult to implement in SEA and LA than 
on the Chinese market, a "zero participation" policy like that implemented by the LAHD is certainly 
not the right approach. 

Eventually, an efficiency analysis only based on some cost/benefit ratios obtained by dividing spent 
resources by number of SMEs advised or SME users or trainees give a relative unfavourable efficiency 
assessment. This is evident by considering the overall high level of ten average costs per unit. But this 
has to be relativized. Firstly, as commented by the authors of the Study "Because IP has an insurance 
function, the possible benefits even for a small number of SMEs which have managed to avoid being 
infringed  and/or  otherwise  involved  in  litigation/enforcement  of  IP  rights  can  outweigh  the 
expenses." Secondly, assessing the efficiency of the IHD in light of productivity ratios or gap analysis 
based  on  too  ambitious  targets  (and  probably  too  numerous  as  well)  seems  irrelevant.  It  is  more 
important to ensure that the resources that the IHD use to settle an organisation, develop tools, get 
expertise,  and  provide  services,  result 
improving  quantitatively  and  qualitatively  their 
performance. The benefits evidenced in the effectiveness analysis show an increased awareness of 
EU  SMEs  of  the  need  to  protect  their  intellectual  assets  before  and  while  entering  the  targeted 
markets.  The  survey  and  interviews  show  that  EU  SMEs  which  have  used  the  services  of  the  IHD 
almost unanimously value the usefulness of their services. After three years of experience under the 
COSME programme, the IHD have been showing effectiveness and should now focus their resources 
to gain in efficiency.  

in 

6.2.3. 

Can  the  IHD  achieve  more 
resources)?  

impact  (within  current  or 

increased 

The grants contributing to financing each IHD entity seem sufficient to achieve the objectives of the 
initiative within the agreed KPIs. Overall, the current IHD entities do not find the budget too low to 
accomplish their tasks168. However, Table 3 above shows that the IHD properly fulfil their targets in 
relation to one  and only  service  i.e. the  helpline. For the remaining services, the  IHD do not  reach 
their  targets.  This  is  not  due  to  lack  of  financial  resources. Within  the  current  budgets,  progresses 
could be made to move closer to the targets.   

This would imply concentrating efforts on:  

  Trainings: by making a better use of resources to develop more modules and deliver more 
actions. Considering the good attendance rate, three times more participants might possibly 
be reached.     

  Promotion: as indicated above the survey shows that an important portion of respondents 
consider that the IHD are insufficiently promoted (more than 50% in LA, between 30 and 40% 
in China and SEA). In addition, promotion would probably be better when the entity in SEA 

168  see  interview  synthesis,  the  only  instance  mentioned  is  that  the  SEAHD,  due  to  budget  limitations,  could  not  cover 
Vietnam with an on-site partner   

 
 
                                                           
and  particularly  in  LA  could  improve  their  partnering  service,  as  the  CNHD  does.  Another 
aspect that could help improvement of promotion would be the publication services. The gap 
between  the  publications  achieved  and  targeted  remains  too  important,  while  the  survey' 
respondents value these services as particularly useful and important. 

  Customer  satisfaction:  while  certain  KPIs  have  been  set  to  measure  the  users'  satisfaction 
levels,  they  are  not  equally  reported  by the  three  IHD  entities.  Putting  into  place  a  shared 
and homogeneous SMEs satisfaction barometer would be an interesting efficiency tool.  

Consequently,  there is no need for additional means in order to achieve more impacts,  however 
the use of the current ones could be optimised as was done for the current helpline services. 

6.2.4. 

Governance:  What  implementation  mode  is  optimal:  grant,  contract  or 
other options? 

A  basic  consideration  for  the  efficiency  of  the  operation  of  the  IHD  is  the  contractual  framework 
within which they operate. Currently the IHD operate based on three grant agreements, one for each 
IHD regional entity, plus an additional grant for the development and maintenance of the common 
web  portal.  As  can  be  seen  from  the  interview  synthesis,  the  EC  informed  about  a  possible  other 
option i.e. a contract  agreement. Question arises  as to whether, to implement the IHD initiative, a 
contractual framework would be better than the on-going grant agreement.   

In case of a grant, the Commission makes a contribution to project carried out by an organisation (i.e. 
the beneficiary) because its activities contribute to the EU policy aims. Such funding is considered in 
particular as typically fit to beneficiaries like Research Centres or University covering a broad range of 
theoretical  projects.  The  application  for  financing  originates  with  the  beneficiary,  who  submits  a 
proposal for support for activities it plans to carry out in response to a Commission call for proposals. 
The  proposal  sets  out  the  specifications  for  the  action  to  be  performed  within the  framework  laid 
down in advance by the Commission. The grant must not have the purpose or effect of producing a 
profit for the beneficiary. A pre-financing (by the Commission) usually takes place prior to (or at the 
start  of)  the  provision  of  the  services169.  Save  in  exceptional  cases,  the  grant  may  not  finance  the 
total costs of the action. Ownership generated by the action (like logo, trademarks, website) as a rule 
remains  with  the  beneficiary.  Due  to  this  financing  scheme,  funding  theoretically  appears  as 
encouraging less a maximised delivery of services, than a payment which takes into account delivery. 
The  grant  option  should  normally  imply  less  control  for  the  Commission  and  more  flexibility  for 
beneficiaries in defining the action. However, other factors in particular the way the implementation 
of the action is actually monitored by the Commission or its agency, have to be taken into account. In 
the  framework  of the  current  evaluation  period,  Article  6 of  the  COSME  Regulation  allows  entities 
established in third countries to participate in actions under the COSME Programme, even when the 
third  country  in  question  does  not.  In  accordance  with  Article  7  profit-making  entities  established 
outside  countries  participating  in  the  COSME  Programme  cannot  receive  any  EU  funding.  This 
limitation  does  not  concern  subcontracting.  Subcontracting  may  cover  a  limited  part  of  the  action 
(not core business and 20% of the eligible costs).  

In case of a contract, it is the provision of the service which generates the payment. Such payment 
per delivery is usually seen as providing incentives  to maximise  delivery. There  is normally no pre-

169  e.g.  for  the  IHD,  under  the  current  grant  agreement,  a  pre-financing  of  EUR  2,256,152  (1,299,459.50  for  CNHD  + 
598,138.50 for LAHD + 358,554 for SEAHD)  

 
 
                                                           
financing,  while  the  remuneration  of  the  contractor  includes  an  element  of  profit.  In  addition, 
ownership  generated  by  the  action  belongs  to  the  contracting  authority.  The  core  services  to  be 
provided are pre-defined in detailed specifications prepared by the Commission (or its agency). There 
is not much room for applicants to propose the type of services or the way of implementing them. 
Such  implementation  form  where  the  contracting  authority  has  the  initiative  and  control  over  the 
action,  while  the  selected  applicant  has  to  concentrate  on  its  delivery,  looks  more  adapted  to 
services offered by profit-making entities. In case of a contract with the EASME Agency, participation 
is  open  to  "all  natural  and  legal  persons  coming  within  the  scope  of  the  Treaties,  as  well  as  to 
international organisations", and to "all natural and legal persons established in a third country which 
has  a  special  agreement  with  the  Union  in  the  field  of  public  procurement  on  the  conditions  laid 
down in that agreement170". Other entities can be subcontractors. Subcontracting is unlimited. 

As  the  IHD  services  contribute  to  the  EC  policy  aim  to  support  EU  SMEs  in  relation  to  IP,  a  grant 
agreement is an option for implementing the corresponding services. To implement the IHD, partners 
located  in  China,  SEA  and  LA  (i.e.  in  non-COSME  countries)  are  absolutely  needed.  Public  entities 
established  in  (non-COSME)  countries  can  participate.  Subcontracting  could  be  opened  to  some 
extent (for non-core tasks) to profit-making entities that are otherwise excluded from EU funding in 
these  countries.  During  the  evaluation  period,  this  implementation  form  appeared  as  specifically 
appropriate for the consortium which has been operating the LAHD. As it includes two universities 
and  an  institute  for  legal  and  technological  studies,  it  has  a  strong  academic  component.  Profit-
making  entities  are,  however,  usually  less  interested  in  participating  in  actions  funded  by  grants 
under the COSME programme, which require a co-financing by the applicants (in the current period 
10%) and does not allow them to generate any profit. In addition, unless an exception to the model 
grant  agreement is created171, ownership of the website or IPR generated by the IHD action would 
normally stays with the beneficiary. This could be seen as problematic to ensure in particular good 
continuity  in  the  service  offered  to  EU  SMEs.  During  the  evaluation  period  for  instance,  different 
approaches  to  the  website  design  generated  difficulties  in  the  transition  from the  predecessor  IPR 
Mercosur helpdesk to the LAHD website.  

The very practical aspect of the services to be offered by the IHD (e.g. providing advice in less than 3 
days,  in  a  jargon-free  language,  or  publications  adapted  to  sectorial  concerns)  requires  a  concrete 
approach  that  might  appear  less  adapted  to  the  grant  agreement  framework  usually  reserved  for 
more  academic  projects.  Similary,  the  need  for  the  Commission  (or  its  agency)  to  detail  the 
specification of the IHD services in a contract could be seen as more burdensome than drafting the 
main lines of a grant  proposal. Resorting to a service  contract could possibly  increase the  range of 
applicants, by not refraining profit-making entities from applying as there is no need for them to co-
finance the action and they can generate profit. Non-EU based entities, or entities from countries like 
China, Vietnam and a number of countries of the IHD, that did not sign the Government Procurement 
Agreement  (GPA)  cannot  be  partners  of  a  consortium.  However,  subcontracting  is  fully  open  to 
them. Eventually, to avoid discontinuity in the IPR website service  offered to EU SMEs,  it is critical 
that any ownership generated by the action remains with the EU, as provided by a service contract.  

170  Third  countries  with  a  special  agreement  with  the  Union  in  the  field  of  public  procurement  are:  Albania,  Bosnia  and 
Herzegovina, the former Yugoslav Republic of Macedonia, Montenegro, Serbia, Iceland, Liechtenstein and Norway   
171 Such exception was actually activated under the current COSME Grants 

 
 
 
 
                                                           
The  question of the  choice  of  implementation  is  closely  related  to  the  extent to which  payment  is 
linked to delivery of certain services or not. While payment per delivery as usual in service contract 
could be seen as the best option to maximise the quality and timely delivery of the IHD services, the 
current grant agreement offers a quite strict link between delivery and payment as well. Payment is 
subject to the approval of the periodic reports. For instance, part of the grant allocated to cover the 
development of a joint website has been disallowed from the staff costs claimed by the beneficiaries 
as  that  key  deliverable  was  submitted  almost  one  year  late  from  the  expected  delivery  date.  This 
factor does not lead to choose one option or the other.  

In any event, the way the implementation actually takes place will depend also on additional factors 
like deliverables and KPIs and, more generally, the way the Commission (or its agency) monitors the 
implementation. Practical experience shows that the efficiency of the operation of the IHD does not 
lead  only  in  the  contractual  framework  within  which  they  operate.  The  choice  between  the  two 
options was discussed within the framework of the evaluation of the European helpdesk, as can be 
seen from the report published in 2014. The interviews run within this previous evaluation revealed a 
clear  administrative  burden  for  both  the  Contractor  and  EASME.  Some  interviewees  passed  the 
blame for this burden on the then service contract because a service contract requires a much more 
'hands-on’ management than is the case with a grant agreement. Some interviewees suggested that 
the  contractual  form  made  it  difficult  to  shift  resources  major  priorities  over  the  period  of  the 
contract. However, there was no consensus on these two points: other interviewees suggested that 
the  form  of  the  agreement  did  not  pose  a  major  problem  and  that  there  was  sufficient  flexibility 
within the current contractual arrangements. For the IHD, the COSME programme requires periodical 
and  detailed  reports  (quarterly,  annual,  and  biannual  interim  reports)  of  the  activity  of  the  IHD 
reviewed  through  7  KPIs  that  match  with  the  specific  actions.  However,  this  compulsory  reporting 
was not qualified as a burden during the interviews.  

The only source of concern was voiced by a member of one of the consortia running the IHD i.e. in 
September  2017,  the  European  Chamber  of  Commerce  in  Vietnam  feared  that,  under  a  contract 
option, as a third country organisation, they might be restricted to a subcontractor position172 having 
a minor role in the project (e.g. their logo might not be visible any more). In any event, there was no 
claim as to a possible better/worst performance of the IHD under one implementation framework or 
the other. 

It is unclear that the efficiency of the IHD would improve with the choice of another implementation 
framework.  In  particular,  there  is  no  evidence  that  the  administrative  burden  linked  to  the 
monitoring  of  the  IHD  would  be  lower/higher  under  one  or  other  implementation  option.  In  any 
event, as already highlighted in the evaluation report of the EU helpdesk, the quality, effectiveness 
and efficiency of the reporting system could be improved by developing a simplified structure [based 
on  an  annual  planning  cycle,  in  which  reports  provide  information  on  progress  in  achieving  longer 
term  objectives,  as  well  as  key  outputs].  Developing  such  a  simplified  reporting  system  can  be 
implemented under one or the other implementation option. 

172  In  the  case  of  COSME  grant  agreements,  profit  entities  based  in  non-COSME  countries  are  totally  excluded  from 
receiving EU funds and hence cannot sign a contract.  

 
 
 
 
                                                           
The choice of an implementation method or another is essentially linked to the internal monitoring 
and management control of the EC. High administrative constraints could indeed divert the IHD from 
focusing  on  their  main  objective.  However,  such  diversion  from  core  task  can  be  limited  by 
reorganising the reporting system, whatever the implementation form.   

In addition, having a third country partner as a subcontractor does not mean automatically that the 
attractive  activities  will  be  kept  by  the  main  contractor  for  themselves  or  that  they  might  even 
outsource activities to others than the third country subcontractor. It is the content of the contract 
which establishes such rules. As it is binding, the  selected contractor will have to justify each time 
they diverge from the contract e.g. when they want to assign some work to another subcontractor. 
On the other hand, a subcontractor will have less financial reporting.  

To conclude, it is unclear based on the evidence collected whether, to implement the IHD initiative, a 
contractual  framework  would  be  better  than  the  on-going  grant  agreement.  This  was  also  the 
conclusion of the authors of the evaluation previously run on the European helpdesk. 

6.2.5.  What factors influenced the efficiency of the results achieved by the IHD?  

The objective of the IHD is to offer services in co-ordination with and in direct response to requests 
from EU SMEs and their representatives (e.g. sectoral, regional, local associations of SMEs, chambers 
of commerce). As already highlighted, companies from the private sectors like lawyers or consultant 
firms can also represent EU SMEs. They act as multipliers by offering to their final clients (i.e. SMEs) a 
broader  service  including  IPR  aspects  that  they  normally  offer  against  payment.  This  could  reveal 
problematic  considering  that  this  companies  earn  money  notably  by  re-selling  to  EU  SMEs 
information or advice that they obtain free of charge from the IHD. This could negatively impact the 
efficiency  of  the  IHD  initiative  in  two  ways.    First  the  grant  is  not  supposed  to  feed  the  on-going 
business  of  the  intermediaries  (which  makes,  for  instance,  around  24%  of  the  survey  sample). 
Second, such a portion of intermediaries could be seen as depriving EU SMEs from a direct and free 
of charge  IPR  management  services from the IHD. Seen from another angle, intermediaries  can be 
perceived  as  an  efficient way to  reach more  SMEs. The  problem  is  not  so much  that  SMEs  rely on 
intermediaries that benefit from IP support from the IHD. It is more that due to the lack of visibility, 
EU SMEs are not  necessarily  aware  that they have the  option to turn directly to the IHD. The final 
objective  is  that  a  maximum  number  of  SMEs  are  directly  or  indirectly  made  capable  of  facing  IP 
protection  issues  in  or  related  to  the  targeted area.  Therefore,  the  fact  that intermediaries  act  as 
multipliers,  can  be  seen  as  an  efficiency  lever,  provided  the  visibility  of  the  IHD  increases 
concurrently thanks to more promotion efforts. 

Another issue that might bear consequences on the efficiency lies in the monitoring the IHD activity. 
The huge amount of KPIs (44 for LA, 47 for China, 48 for SEA) to be reported on, in addition to the 
number  of  periodical  reports  requested  (monthly  reports,  quarterly,  interim  and  final  reports) 
represents a heavy workload that generates costs. Dedicating a part of the time and resources used 
for  these  reporting  tasks  to  the  core  activity  could  probably  benefit more  final  users  i.e.  EU  SMEs. 
Eventually, each IHD entity reports in its own way. This heterogeneous way of reporting, complicates 
any comparison between the three regions results and does not lead to a clear overall picture of the 
IHD  performances.  Additionally,  instead  of  focusing  them  on  the  most  effective  services,  all  these 
KPIs  split  the  activity  and  resources  of  the  IHD  among  a  wide  range  of  services  at  times  costly, 
without significant results (e.g. e-learning, partner bulletins modules, serious game, podcasts). Such a 
dispersion of the means and attention of the IHD is likely to affect its efficiency. It could be advisable 

 
 
to reduce the reporting workload by focusing to some of the most significant KPIs and a uniform 
way of delivering. 

6.2.6.  What  is  the  best  methodology  to  measure  impact  of  the  initiative  in  the 

corporate strategy of its beneficiaries? 

The  key  methodological  challenge  to  evaluate  the  IHD  is  to  determine  to  what  extent  the  service 
provided has an impact the IP corporate strategy of its beneficiaries.  

It could be interesting to create a pilot to look on a closer angle at the impact of the IHD initiative 
on the corporate strategy of their users. Running this pilot could take the form of a panel of users 
that  provide  information  on  an  annual  basis.  The  users  would  be  selected  from  the  list  of  IHD 
registered  users  who  significantly  benefitted  from  the  IHD  support  (e.g.  having  followed  trainings, 
having repetitively asked for advice…) during e.g. the last 18 months.  

Pilot  users would be  requested first  to indicate whether they have  implemented actions, following 
the advice of the IHD to protect or enforce their IP. In a second phase, users would then be asked to 
provide information on e.g. whether the contact with the IHD modified or induced new actions on 
their IPR management, or whether the(se) action(s) that they took following their contact with the 
IHD positively influenced their business situation due to better protection and exploitation of IP. In 
addition, evidence of concrete steps having been taken by these pilot users would be asked for (e.g. 
growth  of  sales,  jobs  created/maintained,  new  products  or  services  developed).  The  impact 
assessment  run  by  the  IHD  themselves  in  2016173  could  serve  as  an  example  of  second  phase 
questions to be asked to pilot users.   

Such  pilot  project  could  eventually  lead  to  make  available  to  EU  SMEs  a  collection  of  IP 
management best practices. 

6.3.  COHERENCE  

To reply the three questions below, there is a need to analyse both external and internal coherence 
of the implementation of IHD initiative, i.e. between its various components and vis-à-vis other EU 
SME support measures as well as other relevant programmes contributing to the internationalisation 
of European businesses and their competitiveness in China, SEA and LA.  

6.3.1. 

How  to  ensure  best  cooperation  among  the  three  regional  entities  of  the 
IHD (i.e. in China the CNHD, in SEA the SEAHD, and in LA the LAHD)?  

  
According to the interviews, collaboration between the three IHD regional entities seems to 
be good, apparently slightly better between the CNHD and the SEAHD than between the LAHD and 
the two others.   

  
Specific mechanisms are in place to foster cooperation like a bi-monthly call with all project 
managers,  an  annual  stakeholder  conference,  and  joint  publications  (e.g.  on  trade  fairs)  when 
possible.  Furthermore,  the  IHD  are  currently  discussing  joint  visibility  events.  Lastly,  the  IHD  refer 
companies  to  each  other  and  promote  each  other’s  services.  For  events  in  Brussels,  one  common 
representative represents the three regional entities. 

173 See footnote 16  

 
 
 
 
                                                           
The IHD view their cooperation as good. According to the survey, from a users’ point of view, the 
IHD are not sufficiently promoted. It is likely that improved cooperation in a common promotion 
strategy could help to increase the impact of the IHD communication vis-à-vis the targeted group. 
Such a strategy could lead to think about common scheduled actions like a joined communication 
with  a  uniform  visual  identity  using  polled  communication  means  and  sharing  the  promotional 
material with each other (in Brussels) when going to events.   

6.3.2. 

How to best coordinate the IHD with other EU SME support measures as 
well as other relevant programmes contributing to the internationalisation 
of European businesses and their competitiveness in China, SEA and LA? 

Coherence with EU actions 

1. Best coordination within the COSME programme 

  
The  interim  evaluation of the  COSME  programme,  performed  before the  evaluation  of the 
IHD, mentions two main actions with potential coherence with the IHD namely the Enterprise Europe 
Network (EEN) and the Cluster Internationalisation actions. The other actions included in the COSME 
programme are either solely aimed at the Single Market (COSME countries) or do not have a strong 
ongoing link with IPR.  

Best coordination with the EEN 
  
The EEN has two types of members: within and outside the COSME countries. The Network 
member within the COSME countries provide the full range of Network services to SMEs from their 
country.  The  Network  members  outside  the  COSME  countries,  also  called  Business  Cooperation 
Centres  (BCCs),  only  provide  the  matchmaking  services  as  well  as  information  about  their  own 
market to Network members from the COSME countries.  

  
The  input  from  the  IHD  within  the  framework  of  the  COSME  evaluation,  shows  that  the 
collaboration  between  the  Network  members  and  the  IHD  is  healthy.  The  EEN  members  can  refer 
their clients to the IHD when they have IP related questions in China, LA and SEA. Symmetrically, the 
IHD can refer to the EEN SMEs in need of support beyond the topic of IP. In such a large network as 
that of the EEN, IP appears as an important issue to manage, to only a small portion of EU SMEs. Only 
3%  of  the  Network  members  spontaneously  mention  the  IHD  in  the  open  question  of  the  survey. 
Within  such  framework,  having  an  IHD  specifically  dedicated  to  SMEs  for  which  IP  issues  appear 
consciously or potentially crucial for a successful internationalisation, sounds coherent. These SMEs 
cannot indeed find such an equivalent IP expertise among the broad range of services offered by the 
EEN. 

  
There  could  be  some  overlaps  with  information  given  by  the  specific  BCCs  category  about 
their  home  market.  It  should  however  be  noted  that  the  BCCs  only  provide  very  basic  market 
information  and  certainly  not  in-depth  IP  advice.  As  often  the  BCCs  do  not  have  dedicated  IPR 
expertise, or at  least not  at  the  same level as the IHD, little  to no indications of overlap are found 
between the activities of the IHD and the BCCs. 

Best coordination with the Cluster Internationalisation actions 
  
include 
The  Cluster  Internationalisation  actions  as  part  of  the  COSME  programme 
international  cluster  projects,  a  web  portal  aimed  at  cluster  collaboration  and  a  few  matchmaking 

 
 
events.  The  international  cluster  projects  are  organised  by  the  European  Strategic  Cluster 
Partnerships  (ESCPs)  and  many  of  them  target  markets  in  China,  LA  and  SEA  and  often  revolve 
around the manufacturing industry for which the topic of IP could potentially be important. The web 
portal i.e. European Cluster Collaboration Platform (ECCP)174 provides European Clusters with a tool 
to present themselves and to find others, to look into the ESCPs and their projects, promote various 
cluster events and to stimulate international cooperation. The website includes dedicated pages per 
country (in particular China, South Korea, Brazil and Colombia, as well as other countries covered by 
the  IHD)  providing  some  baseline  information,  key  facts,  key  organisations,  useful  links  and 
documents.   

A  survey  was  held  amongst  the  cluster  organisations  as  part  of  the  evaluation  of  the  COSME 
programme. The survey results show that the CNHD and LAHD are used by around 10% of the cluster 
organisations and the SEAHD by less than 5% of the cluster organisations. The ECCP website shows 
the  IHD  under  the  label  of  "EU  International  Support  Services"  right  next  to  the  EU  IPR  Helpdesk, 
however the IHD are not shown on the country specific pages.  

  
Overall the cooperation with the cluster activities of the COSME programme is minimal and 
there is no overlap. The cluster projects do not refer to IHD and seldom mention the topic of IPR. The 
IHD  indicate  not  to  be  involved  in  any  of  the  COSME  funded  international  cluster  projects.  The 
marginal role of IPR in the cluster projects can be questioned in terms of coherence, as many of them 
are technologically advanced projects for which IP protection could potentially become a key issue. 
The  IHD  indicate  that  a  better  cooperation  with  the  Cluster  Internationalisation  actions  will  be  a 
priority for the future.  

2. Coherence with other EU actions 
  
Other relevant services provided at EU level are the European IPR helpdesk, IPORTA / VIP for 
SMEs.  The  other  activities  are  focused  on  the  internal  market,  while  the  IHD  provide  the  external 
arm. Since the other activities are mostly inward looking, the coherence seems good.  

  
The IP key project is complementary to the IHD both in China and in SEA and is expected to 
be of similar complementary in LA when the new IP key project becomes operational in spring 2018.  

In  the  view  of  the  IHD  members  interviewed,  cooperation  can  be  increased  with  the  EU 

  
delegations and with EU funded projects (especially in SEA and LA).  

  

Potential synergies with similar IHD services provided by other entities 

1. IPR Services provided by MS  

  
11 services have been identified with origin in one of the MSs and usually funded by them, 
having a dedicated IP focus and covering one or more of the regions China, SEA or LA. These services 
are provided, in large MSs only and by IP attachés (France, UK, Germany and Italy to a lesser degree) 
or  in  some  cases,  by  trade  organisations  (like  the  German  AHK  or  the  Italian  ITA  IPR  Desks)  or 
business  associations  (like business  Sweden).  Most  of  these  services,  like  the  IHD  ones,  provide  to 

174 https://www.clustercollaboration.eu/  

 
 
 
                                                           
some degree first line support and aim at raising overall awareness on IP matters while not excluding 
specific IP instruments. The main differences between the IHD and these services are that the latter 
focus on companies of all sizes and originating from the respective MS only. 

  
For similar services in 8 MSs which provide support in IP matters in the same third countries 
regions,  a  deeper  analysis  was  undertaken.  These  similar  services  do  not  usually  overlap  with  IHD 
services.  They  act  as  intermediaries  by  sign-posting  their  beneficiaries  to  the  IHD  helplines  or 
publications, participating or hosting joined events, or contributing to publications incl. newsletter. 
While cooperation with IP attachés is valued among the strongest, Chambers of commerce appear to 
cooperate less.  

  
Synergy  works  better  with  the  longer  established  CNHD  than  with  the  SEAHD  of  the  LAHD 
where cooperation often only just started. In China, the CNHD is apparently known to all interviewed 
stakeholders  both  in  China  and  in  Europe  (French  INPI  attachés,  UK  IP  attachés,  the  German  AHK 
Innovation Centre Denmark…) are either in direct contact, or refer to their publications, cooperates 
in events. In SEA, stakeholders know the SEAHD, although apparently to a lesser extent in particular 
in  Europe.  Cooperation  there  is  good  with  IP  attachés,  but  less  with  the  three  chambers  of 
commerce. Strongest links had been built with one of them, which were eventually disrupted. In LA, 
similar services know the IHD services but do not cooperate or only in a few webinars or events. The 
French INPI attaché in Brazil is in contact with the EU delegation but  not with the IHD, which is of 
concern. In this region, there is apparently room for more cooperation with other similar services in 
particular in this geographical area with the CIBEPYME network175 that plans to provide more direct 
IP support services to SMEs in the future. 

  

2. Services on IPR provided by private entities 

  
Private  services  include  IP  law  firms  and  similar  services.  Private  services  are  focussed  on 
legal  support  and  consultancy  services  whereas  the  IHD  limit  themselves  to  providing  first-line 
support and raising awareness. Therefore, if a company was made aware of the importance of IPR 
and  has  been  provided  some  guidance  on  IPR  types  and  how  to  register  and  protect  them,  the 
company  still  must  get  in  touch  with  a  legal  service  provider  to  register  IPR.  To  this  end,  the  IHD, 
when  legally  allowed,  provide  guidance  to  registered  professionals  in  the  relevant  regions.  Private 
services themselves consider the IHD not as competitors, but as potential intermediary for clients. 

  
As evidenced both by the finding of the survey and interviews, most SMEs even the micro-
sized ones, use IP professionals (lawyers, patent attorneys) to assist them with their IP management.   

Conclusion on coherence 

To maintain coherence with all the public or private actors likely to provide similar IP services, there 
are several avenues for improvement:  

1°)  Internal  cooperation  between  the  three  IHD  entities:  To  reduce  the  heterogeneous  visibility  of 
the  IHD  entities  vis-à-vis  stakeholders  notably  in Europe,  the  IHD  could  be  encouraged  to  organise 

175 The CIBEPYME platform (Platforma Iberoamericana de Propriedad Industrial Dirigida a Empresas) provides IPR advice 
tailored to SMEs from Spain, Portugal and selected Latin-American countries 

 
 
 
                                                           
common  actions  by  polling  resources,  tools  and  information  whenever  possible  like  e.g.  sharing 
promotional materials with each other (in Brussels) when going to events.  

2°)  Coherence  with  other  networking  initiatives  within  the  COSME  programme  (EEN,  BBCs, 
internationalisation clusters etc.):  

  The IHD should ensure that they appear as the IP experts towards which EU SMEs must be 
redirected. In that regards, the IHD should ensure good professional communication around 
their services, and that EU networks gathering SMEs (EEN, BBCs, internationalisation clusters 
etc.) make them more visible (e.g. for example, by requesting more support from the BBCs or 
asking the EEN for an access to their events calendar). The IHD would have a vested interest 
in increasingly promoting within these networks, the IP protection approach as both a major 
risk and a key success factor for SMEs having valuable IP assets. 

  The  low  synergy  with  the  international  cluster  projects  seems  unhealthy.  There  is  no  valid 
reason, not to develop it. A better cooperation could benefit both partners. A first step could 
be to include the IHD in the country specific pages of ECCP website. 

3°) Cooperation with other similar services: 

  Maintain  good  relationship  with  IP  attachés  or  CIBEPYME  by  following  the  example  of  the 

long-proven cooperation in China 

  Enlarge collaboration with chambers of commerce for all three regions in particular in LA  

  always insist on the SME focus of the IHD services, as a positive differentiating criterion 

  highlight  the  IP  expertise as an added-value  (easy to access and free  of charge)  of the IHD 

services and a welcomed addition to the attachés own expertise. 

6.4.  EU VALUE ADDED  

6.4.1. 

Are the IHD providing value added on top of existing services provided by 
MS,  by  the  EU  or  by  private  service  providers?  In  particular  are  they 
complementing the core trade promotion activities of MS? 

From the views expressed by a majority of survey respondents, the added value of the service seems 
to be, for users in the provision of impartial, free of charge, pan-European as well as (third country) 
local advice on IPR. It is difficult to conceive that these values would have been achievable without 
the IHD, even when the MSs have themselves invested in IPR support.  

Several MS (UK, DE, FR and IT) provide to their national SMEs similar IPR services in situ within the 
framework of their national globalisation policy support. These services are usually provided free of 
charge176. There is also a bilateral initiative developed between Spain and South America (CIBEPYME 
platform).  The  characteristic  features  of  the  service  provided  coincide  to  a  certain  extent  with  the 
one  offered  by  the  IHD.  However,  interviews  suggest  that  while  larger  MS  can  provide  similar 
services to SMEs in their national language, smaller MS cannot. In addition, the quality of advice and 

176 One MS apart i.e. Germany   

 
 
 
                                                           
depth  of  services  provided  at  national  or  bilateral  level  varies  from  one  MS  to  the  other.  (Source: 
Desk research and interview):  

  For instance, the INPI (French Industrial Property Office) provides three support services to 
French SMEs in their IP international strategy ("booster", "pass" and "master-class"). French 
companies  can  access  the  INPI  attachés,  i.e.  the  11  experts  located  in  different  countries 
(including  LA,  China  and  SEA).  Additionally,  the  INPI  attachés  produce  fact  sheets  on  IPR 
matters  in  the  countries  they  cover  and  give  first  level  advice.  French  companies  can  also 
benefit from support and can have access to a large network of experts in the context of the 
scheme "SME France without counterfeit".  

  The  Italian  Trade  Agency  together  with  the  Italian  Ministry  of  Foreign  Affairs  and 
International  Cooperation  (MAECI)  maintains  IPR  Desks  at  the  embassies  of  selected 
countries. The service covers countries such as Russia, the US, China, Turkey and Japan. The 
SME  associations’  internationalisation  department  acts  as  a  relay  and  forwards  companies’ 
requests to the relevant IPR Desk. 

  The  Delegations  of  German  Industry  &  Commerce  in  Beijing,  Shanghai  and  Guangzhou 
provide legal counselling to German companies active in China. The AHK informs about the 
Chinese  IPR  -  laws  and  regulations,  give  customer-tailored  advice  on  how  to  manage  and 
especially how to protect IP and also undertake all formalities for registering domain names, 
trademarks and patents in China. Furthermore, the AHK branches can provide support if IPR-
infringement  occurred  to  a  German  company.  This  includes  writing  letters  to  the  relevant 
authorities in China, lobbying, advising on the best reaction and seeing the company through 
the proceedings in case the infringement is brought to court (serving as the contact point to 
the  Chinese  attorney).  Among  the  sites  where  the  AHK  is  active,  there  are  considerable 
differences  in  the  services  provided.  According  to  the  interviewee,  in  Beijing,  the  focus  is 
mostly on trademark registrations, while  the service  in Shanghai is covering all types of IP. 
The  main  difference  between  the  services  is  seen  in  the  language,  but  also  in  services  not 
offered  by  the  AHK  like  mediations,  and  for  the  Beijing  AHK,  the  focus  of  activities  on  the 
registration of trademarks. Eventually, their services are not provided free of charge. 

  The  CIBEPYME  platform  is  the  result  of  the  Ibero-American  Industrial Property  Programme 
(IBEPI)  implemented  in  2009  to  provide  IPR  support  to  Ibero-American  SMEs  that  plan  to 
internationalise.  The  platform  informs  and  assists  SMEs  about  the  protection  of  their 
investment,  trademark,  designs,  and  other  IPR  forms.  The  platform  covers  the  countries 
Argentina,  Colombia,  Ecuador,  Mexico,  Peru,  Dominican  Republic,  Brazil,  Costa  Rica, 
Guatemala, Paraguay, Portugal, Spain and Uruguay.  

  
With these national (or regional in case of CYBEPYME) support, SMEs that intend to export 
their products or services towards geographical areas covered by the IHD, can benefit from services 
comparable  to  the  ones  offered  by  the  IHD.  They  have  access  to  information,  to  advices,  and 
sometimes also to trainings. These services exist for companies of the respective MS. These MSs are 
usually  among  the  largest.  For  SMEs  from other  MSs,  a mere  access  to  similar information  reveals 
more difficult and usually means to have to resort to costly experts. For these SMEs, the added-value 
of  the  IHD  is  valuable.  They  would  not  normally  benefit  from  equivalent  expert  advices  for  free. 
Could  this  mean  that  while  SMEs  from  the  largest  MS  obtain  satisfaction  trough  their  own  MS 

 
 
services,  the  added-value  of  the  IHD  is  exclusively  reserved  for  SMEs  from  the  other  MSs?  The 
evidence gathered tends to show that this is not the case. First of all, the, albeit, limited number of 
SMEs  having responded to the  survey include  SMEs  from 14 MS among which the largest MSs are 
well  represented.  In  addition,  national/regional  support  services  do  not  usually  see  the  IHD  as  a 
competitor.  To  the  opposite,  the  IHD  are  normally  seen  as  a  body  gathering  expert  information 
means  and  advices  in  the  service  of  EU  SMEs.  There  is  no  indication  that  the  national  or  regional 
services  would  prevent  their  SMEs  from  accessing  the  IHD.  On  the  contrary,  national  or  regional 
services often ask the IHD for cooperation and/or direct their respective SMEs towards the IHD. One 
interview  of  the  Study  highlights  that  the  Beijing  (German)  AHK  knows  the  CNHD  very  well  and 
cooperates especially on event organisation and by referring companies to the homepage of the IHD. 
This evidences that the IHD expertise is acknowledged. Despite these national or bilateral initiatives, 
the IHD adds value at EU level by developing and broadening this expertise to all EU SMEs. In the 
absence of the  IHD, SMEs in several MSs would not benefit from IPR support  services. In addition, 
allowing  such  a  free  access  to  these  services  for  all  EU  SMEs  is  easier  by  pooling  resources  at 
European level.  

From  a  qualitative  point  of  view,  as  evidenced  from  the  survey  run,  users  seem  satisfied  with  the 
services provided by the IHD. They find them essential while both planning to expand their business 
activity in the target countries and once they have entered them. Survey respondents highly  value 
the relevance, speediness, and depth of the information provided by the IHD, as well as the quality of 
their advice. Therefore, it is very important to ensure this level of quality in an homogeneous way at 
EU level. In order that any SME in any EU MS have access to an equivalent quality of IPR service, it is 
of specific value to be able to offer them an IPR one-stop-shop like the IHD to accompany them in 
their  globalisation  projects.  This  is  what  gives  special  add  valued  to  the  European  IHD  by  making 
them more effective or less costly than services provided directly by MSs or by experts. 

In  addition,  there  is  apparently  an  EU-added  value  in  terms  of  skills  development  for  EU  SMEs. 
Focusing resources on the IHD maintains and increases  the level of expertise  that SMEs  need. This 
requires the IHD to adapt trainings, helpline, publications, etc. to the highest standards. As a result, 
the  IHD  are  acknowledged  as  expert  in  this  area,  available  to  any  EU  SME  in  any  MS.  The  IHD  is 
unique in its dedication to the type of support it provides. It should also be noted that the form of 
the assistance provided, and the professionalism of the delivery generally add to the capacities of the 
organisations assisted and consequently add to their longer-term viability. 

In the end, designing a common EU website, grouping trainings and publications, and sharing their 
costs, is an added-value from an economic point of view. The IHD offers services that could not be 
easily provided individually by each MS and allow to pool expertise to be available to any EU SME. 
Such  economies  of  scale  also  enable  to  customise  the  training  or  advice  to  specific  sectors  or 
business  models.  The  service  also  avoids  duplication  of  technical  contacts  with  relevant  countries 
administrations to signpost SMEs. 

6.4.2.  What  would  be  the  most  likely  consequences  of  stopping  or  withdrawing 

the measure?  

One  question  from  the  user  survey  asked  the  survey'  respondents  about  the  consequences  of 
stopping or withdrawing the IHD for their organisation. Most respondents clearly agree that it would 
increase their overall workload. An important portion of them agree that it would generate negative 

 
 
consequences  like  unwanted  know-how  flows  (52%),  increase  of  IPR  costs  (51%),  or  more 
infringement situations (48%). This shows that, for their current users, the IHD are a key element to 
solve  IP  matters  and  secure  their  IPR  when  they  plan  to  expand  internationally  or  are  yet  trading 
outside the EU.     

7.   CONCLUSION 

The aim of the IHD to strengthen European SME’ growth and competitiveness is directly aligned with 
the policy objective of the Juncker plan to boost for Jobs, Growth & Investment. "The protection of 
Intellectual  Property  Rights  in  third  countries  is  crucial  for  promoting  innovation  and  business  of 
Europeans  SMEs  abroad.  By  facilitating  and  supporting  companies  to  engage  in  international 
business,  the  helpdesks  indirectly  stimulate  their  growth  and  competitiveness”.  The  present 
evaluation  confirms  that  the  IHD  initiative  help  export-oriented  European  SMEs  benefit  from  the 
necessary tools to protect their IP. To achieve this, the IHD improve the EU SMEs' awareness of the 
importance  to  take  appropriate  formal  and  legal  steps  in  order  to  reduce  counterfeiting  and 
infringement risks. Through providing suitable services (e.g. helpline, website, publications, training), 
the IHD allow EU SMEs to receive a complete and detailed information, practical recommendations 
and skills development. Offering thus, through the IHD, a free of charge access to expert information 
and tailored training, the EU brings support acknowledged by the SMEs which employ them. The IHD 
take place in a coherent manner inside the range of services offered by the EU to companies through 
the COSME programme. IP protection is a crucial subject for companies whose added value is based 
on  their  intellectual  assets.  However  it  is  not  among  the  major  themes  addressed  by  other 
networking initiatives of the COSME programme (e.g. EEN or the Cluster Internationalisation actions) 
because  the  expert  knowledge  the  IHD  brings  interests  a  limited  amount  of  SMEs.  The  specific 
mission that the IHD offers to these EU SMEs does not normally clash with national/regional entities 
providing comparable services in some MS, either directly on their territories or via their "détachés" 
in  third  countries.  On  the  contrary,  national  administrations,  IP  civil  organisms,  chambers  of 
commerce  are  inclined  to  collaborate  with  the  IHD,  e.g.  by  translating  their  publications,  referring 
their own SMEs to the IHD, or requesting them to organise common events. It is difficult to estimate 
the  impact  of  the  IHD  on  its  specific  objective  to  reduce  the  risk  of  counterfeit  products  from 
entering the European market. However, evidence from the users' survey177 and interviews indicate 
that  the  IHD  services  appear  as  important  and  useful  to  them.  They  acknowledge  that  the  IHD 
contribute to make  them aware  of  the  challenges  and  risks  linked  to  IP  protection  in  the  targeted 
third  countries.  Furthermore,  a  percentage  of  survey'  respondents  mention  that  the  contact  that 
they had with the IHD did impact their IP management. These results are positive.  They show that 
support provided to EU SMEs in this area bear fruit. Therefore, there are indications that the IHD is 
overall effective in improving the knowledge of IP protection requirements in third countries which is 
a  first  step  to  reduce  the  risk  of  counterfeit  products  entering  the  EU  market.  A  prerequisite  is 
however that the IHD services obtain sufficient visibility among the concerned EU SMEs.  

The  IHD  should  seek  in  addition  to  be  more  efficient  in  achieving  the  specific  actions.  This  is  very 
important to progress towards the achievement of the overall objectives of the COSME Programme 

177 70 users and 40 potential users replied to the surveys 

 
 
 
                                                           
and the specific objective of reducing the risk of counterfeit products entering the European market 
from the concerned markets. At the end of the period covered by  the evaluation period, the three 
regional entities  of the  IHD do not  show  an equivalent level of efficiency. This limits the impact of 
their services on SMEs. While the IHD offer is quite accessible, due to the lack of a sufficient visibility 
of  the  IHD,  an  important  number  of  potential  users  of  their  services  are  not  reached.  Without 
support  at  the  appropriate  time,  such  potential  users  are  likely  to  launch  into  internationalisation 
projects without (sufficiently) mastering the risks linked to IP protection. The IHD initiative appears 
therefore as a relevant, coherent, effective policy adding a real value to EU policy in favour of SMEs. 
Its efficiency needs to improve in order that the resources allocated to the IHD are used optimally to 
provide the best expected services to the largest possible number of EU SMEs concerned. The main 
outstanding concerns are listed below:  

 – The strategy to reach the critical mass of SMEs needs to be improved 

The IHD communication strategy to reach these EU SME is mainly based on cooperation with 
stakeholders and attendance to events where the IHD might meet their target users. Based 
on this strategy, the IHD get potential users to know them by using intermediaries to show 
themselves  or  by  being  embedded  in  websites  or  communication  portals.  This  "pull" 
communication results in a limited knowledge of the IHD services among the potential SMEs 
doing business in the concerned regions. Additional more proactive and direct approaches, is 
missing. The  three  IHD regional entities do  not  promote sufficiently  their activity via direct 
contact, i.e. in a "push communication", aimed at EU SMEs from the manufacturing area or 
having a high technological added value and internationalisation projects in mind, for which 
IP protection might represent a real challenge.  

– The reporting mechanism for KPIs is burdensome and not sufficiently quality oriented 

To measure the progress achieved by the IHD initiative, the grant agreements impose on the 
beneficiary organisation (i.e. the consortium managing the three regional entities of the IHD) 
to  provide  follow-up  and  inform  about  an  important  amount  of  detailed  performance 
indicators  (e.g.  webinars,  country  factsheets,  case  studies,  serious  game,  video  podcasts, 
training modules, etc.). Most of them deal with short-term outcomes and are more oriented 
towards  quantity  than  quality  results.  By  concentrating  on  tasks  execution  under  the 
pressure of such a high number of operational targets, there is a risk to draw resources from 
the  core  activity  which  is to  support  and  raise  the  awareness of  the  highest  number of EU 
SMEs on IP protection. As evidenced, the IHD report only a few pieces of information about 
possible  progress  in  the  number  of  their  users,  or  the  satisfaction  level  reached  by  their 
services. Few data appear on the nature and frequency of the requests from SMEs as well. 
Getting  information  about  these  elements  appear  as  very  important  to  follow-up  on  the 
performances of the IHD activity. The reporting system does not encourage enough the IHD 
to  progress  towards  more  efficiency.  The  IHD  are  bound  to  follow  up  on  around  50 
indicators.  With  such  a  huge  amount  of  detailed  objectives,  the  IHD  activity  lose  itself  by 
achieving a lot of tasks instead of being focussed on selected keys objectives and challenged 
on relevant action plans. 

– The IHD's visibility is quite poor, pro-active external promotion/communication plans are missing 

 
 
Increasing  the  number  of  SMEs  resorting  to  the  services  of  the  IHD  is  crucial.  Improving 
efficiency  is  useless  when  it  benefits  only  a  restricted  number  of  the  targeted  EU  SMEs, 
whereas the rest remains ignorant of the benefits the service offer could provide them. Too 
low visibility has been cited as one major issue of the three regional entities of the IHD due 
to an insufficient promotion effort. It is a more paramount issue for the younger regional 
entities of the IHD than for the CNHD. Without annual communication plans grouping and 
planning key partnership actions, participations to events, presentation in conferences and 
seminars,  insertions  in  websites  or  partners  portals,  etc.  the  IHD  cannot  improve  their 
visibility and cooperation with key stakeholders and SME intermediaries within Europe and 
in the operating regions.  

– Lack of reliable indicators measuring the effectiveness of the IHD 

The  satisfaction  indexes  are  rarely  filled  in  the  IHD  progress  reports.  The  lack  of  reliable 
satisfaction indicators on helpline, trainings and publications services hampers the follow up 
of the evolution of the IHD quality of service perceived by their users.  

– Continue funding the IHD for the three regions and consider extending this type of service to 
India 

The  results  of  the  relevance  and  of  the  effectiveness  of  the  IHD  support  the  notion  to 
continue  its  funding.  The  topic  is  relevant,  and  there  is  apparently  a  sizable  number  of  EU 
SMEs  which  do  and  would  use  the  service.  According  to  the  available  information,  the 
benefits seem particularly outspoken for micro-enterprises therefore the SME focus should 
be  maintained.  Furthermore,  the  IHD  interact  mostly  coherently  with  IP  support  services 
abroad  from  MSs  (when  such  services  exist).  EU  SMEs  seem  interested  in  receiving  similar 
support  for  other  markets,  in  particular  India,  given  the  feedback,  as  well  as  the  growing 
importance of India as an economy both for selling and producing goods. 

8.  Annexes 

Annex 1: Procedural information 

6. 

LEAD DG, DeCIDE PLANNING/CWP REFERENCES 

Lead DG: Directorate-General Internal Market, Industry, Entrepreneurship & SMEs (GROW) 

Work programme: Programme for the Competitiveness of the Enterprises and Small and Medium-
sized Enterprises (COSME) 
Specific measure: "COS-WP2014-2-05 China, Latin America and South-East Asia IPR SME Helpdesks." 

7.  ORGANISATION AND TIMING 

The evaluation of the IHD was steered by the members of an Inter Service Steering Group involving 
various services of the Commission. Meetings of the Interservice steering Group took place on 2 
December 2016, 28 April 2017, 19 July 2017 (kick-off meeting for the external study), 23 October 
2017 and 24 November 2017.   

 
 
 
Invited DGs and services: DEVCO, FPI, TRADE, RTD, SG and EASME 

8. 

EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

In addition to the internet based open public consultation of 12 weeks carried on for the overall 
COSME programme a consultation was run in the framework of the Study run by the external 
consultant. The survey with users and non-users of the IHD was executed electronically between 
August 2017 and September 2017.  

9. 

CONSULTATION OF THE RSB (IF APPLICABLE) 

(Not applicable)  

10.  EVIDENCE, SOURCES AND QUALITY 

The following evidence has been used in the evaluation: 
COSME Regulation (EU) No 1287/2013 
Competitiveness and Innovation Programme (CIP) 
Competitiveness of SMEs (COSME) programme  
For China the CIP work programme 2010 (ENT/CIP/10/B/N02C00) and 2013 
(ENT/CIP/13/B/N02C00),  
For LA and SEA the CIP work programme 2012 (ENT/CIP/12/B/N04C00) 
2012 Grant Agreement for the SEA IPR SME HD  
2013 Grant Agreement for the LA IPR SME HD  
2015 Grant agreement for the China IPR SME HD 
2015 Grant Agreement for the SEA IPR SME HD 
2015 Grant agreement for the LA IPR SME HD i.e. 

  Original Grant Agreement 
  Annexe 1 to original Grant Agreement 
  Amended Grant Agreement 

Information/link to agencies, EU committee, main stakeholders: EASME 

 
  External web site of relevance, databases etc. 

 
 
 
An external consultant  (Technopolis Group) run a study to partially support the evaluation on behalf 
of  the  Directorate-General  Internal  Market,  Industry,  Entrepreneurship  &  SMEs,  Directorate  F  – 
Innovation and Advanced Manufacturing of the European Commission. This study was run within the 
Framework Contract between Technopolis Group and Internal Market, Industry, Entrepreneurship & 
SMEs  (ENTR/172/PP/2012/FC/LOT  4  –  Evaluation of EU programmes,  legislation, policies  and other 
activities).  

Three independent experts reviewed and gave an opinion on the draft final report prepared by the 
contractor and the current staff working document of the Commission services, and presented their 
views178 on the relevance and coherence of the IHD initiative. 

178  U:\IPR  Helpdesks\02.  International  helpdesks\01.  Intl  HD  2017  evaluation\Experts  opinion\ExpertGroup-Report-

final.docx