Interim Evaluation of the ECSEL Joint Undertaking (2014-2016) operating under Horizon 2020 : executive summary.
Date: 2017-10-18
Institutions: Directorate-General for Communications Networks, Content and Technology
Subjects: EU research policy, Framework Programme for Research and Development, business policy, computer systems, cybernetics, electronic component, innovation, nanotechnology, new technology, report, research and development

FR 

Interim Evaluation of the 
ECSEL Joint Undertaking 
(2014-2016) Operating 
under Horizon 2020 

EXECUTIVE SUMMARY 
A report prepared for the European Commission 
DG Communications Networks, Content & Technology by 

Haydn Thompson, Rapporteur 
Expert group 
Emilio Lora-Tamayo, Chairman  
Werner Damm  
Jean-Luc Dormoy  
Leonard Hobbs 
Margriet Jansz 
Tomasz Kosmider  
Wolfgang Pribyl 

June 2017 

Digital 
Single 
Market 

 
 
 
 
 
 
 
 
DISCLAIMER 

By the European Commission, Directorate-General of Communications Networks, Content & Technology.  

The information and views set out in this publication are those of the author(s) and do not necessarily reflect the 
official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this 
study.  Neither  the  Commission  nor  any person  acting on  the  Commission’s behalf  may be  held responsible  for 
the use which may be made of the information contained therein. 

ISBN 978-92-79-69626-8 
doi:10.2759/517786 

Copyright © 2017 – European Union. All rights reserved. Certain parts are licensed under conditions to the EU.  
Reproduction is authorised provided the source is acknowledged.  

2 

 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3 

 
 
 
1. EXECUTIVE SUMMARY 
The ECSEL JU is strategically, excellently positioned to have strong impact on strengthening European 
industry in key application domains which generate billions of Euros turnover for Europe and 10’s of 
millions of jobs across Europe. It provides a unique opportunity for speeding up innovation through 
tight  integration  of  start-ups,  SMEs,  LEs,  and  RTOs  along  the  complete  vertical  chain  from  new 
applications  to  silicon  and  push  pre-standardisation  activities  of  Platforms.  McKinsey  [1]  estimates 
that the on-going digitisation of industry will potentially add €1 trillion to the GDP in Europe. Sectors 
that crucially rely on the Electronic Components and Systems market include: 

Automotive  -  The  EU  is  among  the  world’s  biggest  producers  of  motor  vehicles,  and  the  sector 
represents  the  largest  private  investor  in  research  and  development  (R&D)  within  Europe.  The  sector 
provides jobs for 12 million people and accounts for 4% of the EU’s GDP. Manufacturing accounts for 3 
million jobs, sales and maintenance for 4.3 million, and transport for 4.8 million. The global car fleet is 
predicted to double from currently 800 million vehicles to over 1.6 billion vehicles by 2030. Markets and 
Markets  predicts  that  the  global  traffic  management  market  will  grow  from  $4.12  billion  in  2015  to 
$17.64 billion by 2020 and the self-driving car market will grow from $42 billion in 2025 to $77 billion by 
2035. 
Rail - The overall rail sector in the EU, including the rail operators and infrastructure managers, employs 
approximately 1.8 million people with an estimated 817,000 dependent individuals. The European rail 
supply industry employs nearly 400,000 people and is a top exporter, accounting for nearly half of the 
world market for rail products with a market share of 84% in Europe and a total production value of €40 
billion  (2010).  Markets  and  Markets  predicts  that  the  railway  management  system  market  will  grow 
from $29.27 billion in 2016 to $57.88 billion by 2021. 
Aerospace - The European aerospace  industry is a  world leader in the production of civil and military 
aircraft,  helicopters,  drones,  aero-engines,  and  equipment,  exporting  them  all  over  the  world. 
Aerospace within the EU provides more than 500,000 jobs and generated a turnover of €140 billion in 
2013.  The  commercial  aircraft  market  is  expected  to  grow  steadily  to  2035.  The  aircraft  flight  control 
system market projected to grow from $11.85 billion in 2016 to $16.59 billion by 2021, and the aircraft 
health  monitoring  systems  market  to  grow  from  $3.43  billion  in  2016,  to  $4.71  billion  by  2021.  The 
Unmanned Aerial Vehicle market was estimated to be $13.22 billion in 2016 and is projected to reach 
$28.27  billion  by  2022  with  opportunities  in  software  ($12.33  billion  by  2022)  and  services  ($18.02 
billion by 2022). The Air Traffic Management (ATM) market is projected to grow from $50.01 billion in 
2016 to $97.30 billion by 2022. 
Manufacturing - The manufacturing sector accounts for 15.0% GDP and provides around 33 million jobs 
in Europe. Europe is a front runner in manufacturing excellence with the vision of smart and connected 
factories swiftly becoming a  reality.  The industrial control and factory automation market, comprising 
control  system  manufacturers,  field  components  manufacturers,  system  integrators,  and  software 
manufacturers,  is  projected  to  reach  $153.30  billion  by  2022.  By  2025  additive  manufacturing  is 
expected  to  create  a  €6.3  billion  opportunity  in the  consumer  electronics,  automotive  and  aerospace 
industries.  
Health - Health care and long-term care expenditure accounted for 8.7% of GDP and about 15% of total 
government expenditure in the EU in 2015. Spending is rising faster than GDP and it is estimated that it 
will reach 16% of GDP by 2020 in OECD countries. The health sector accounts for 10% of all employment 
and  is  expected  to  grow  by  a  further  1.8  million  jobs  up  to  2025.  Life  expectancy  currently  increases 
with “one weekend per week” in Europe. The ageing population and prevalence of chronic diseases will 
increase public health and care budgets significantly due to the need to provide long-term care driving 
the need for new solutions. The healthcare IT market is projected to reach $280.25 billion by 2021 from 
$134.25 billion in 2016. The global medical device connectivity market is projected to reach $1.34 billion 
by 2021 and the telehealth market is projected to reach $9.35 billion by 2021. 
Table 1 Market Opportunities for Electronic Components and Systems (Source: THHINK [2]) 

The tri-partite organisation of ECSEL allows strategic alignment of Member States, Industry and the 
European Commission to push its key mission, thus supporting the findings of the Digitising European 
Industries  Summit  in  Rome  in  March,  with  the  expectation  of  mobilising  a  total  investment  of  €5 
billion. Already in the  period 2014-2016 €1956.8 million of funding and over 1000 companies have 

4 

 
 
been engaged in implementing this mission. Already ECSEL has been highly successful in supporting 
the interests of the micro/nanoelectronics sector such as in supporting the development of FDSOI, a 
key  advanced  low-power  technology,  and  in  keeping  production  capabilities  for  advanced  silicon 
processes  as well as the resulting know-how  in Europe. Projects  such as CRYSTAL  demonstrate the 
potential for creating success stories in the higher system levels, nowadays referred to as (Systems 
of)  Cyber-Physical  Systems,  underlying  Smart  Mobility,  Smart  Energy,  Smart  Home,  Smart  Health, 
Smart  Production  (Industry  4.0)  or  Smart  Cities.  ECSEL  has  already  taken  clear  steps  towards 
achieving its key mission.  

To  unfold  its  potential  and  fully  achieve  its  key  mission  of  supporting  the  Digitisation  of  European 
Industry ECSEL should address the following critical points: 

a.  There is a need to develop a global overarching strategy supported by commitment from 
all  parties  involved  for  Electronic  Components  and  Systems  that  addresses  the  vertical 
integration chain 

b.  There is a need to synchronise national activities, harmonise participation rules, funding 

rates and procedures 

c.  There is a need to place greater emphasis and target resources on coverage of the value 

d. 

chain, particularly with respect to systems 
Instruments  should  be  created  targeted  at  encouraging  more  SME  and  start-up 
participation in the ECSEL community in order to more closely meet the goals sets within 
H2020 (20% of allocated budget allocated to SMEs) 

e.  There is a need to put in place appropriate metrics and compulsory follow up to assess 
the impact of projects to justify funding commitments from the EU, Member States and 
Industry 

In addressing these critical points we recommend: 

a.  Strengthening  the  involvement  of  system  industries,  in  particular,  by  ensuring  that 

Lighthouse projects are driven by systems companies 

b.  At  the  highest  political  level  a  discussion  should  be  initiated  to  harmonise  and 
synchronise  the  Member  State  participation  rules,  funding  rates  and  procedures 
wherever possible, adopting best practice as the guiding principle, with the objective of 
introducing harmonised rules for the next framework programme 

c.  Development of an industry driven top-down overarching strategy concentrating on the 
integration of Cyber-Physical Systems (CPS), smart systems and electronics with a focus 
on the key application domains such as automotive, health, energy, etc. 

d.  Capitalise on the innovation capabilities of start-ups and SMEs by fully integrating them 
into  innovation  ecosystems  for  the  key  targeted  market  segments  organised  around 
Lighthouse projects, such as by developing open platforms 

e.  Although  it  is  clear  that  the  ECSEL  outcomes  are  resulting  in  impact,  in  order  to  more 
accurately  measure  impact  and  success  of  projects  appropriate  metrics  should  be 
collected during project execution and also subsequently after a project is completed. In 
order  to  benchmark the  results  at  an  international  level  it  is  important  to  also  provide 
assessment from international experts. 

5 

 
 
 
 
 
 
 
1.1 Extended Executive Summary 

This  report  presents  the  results  of  an  interim  evaluation  of  the  ECSEL  JU  to  assess  progress  in  the 
period 2014-2016. The ECSEL JU was set up in 2014 with the remit to keep Europe at the forefront 
of technology development in the area of Electronic Components and Systems. ECSEL combines two 
previous JUs that addressed the areas of embedded systems (ARTEMIS) and nanoelectronics (ENIAC). 
Additionally, to gain a more complete integration of the value chain, the EPoSS European Technology 
Platform on Smart Systems Integration was  also combined into ECSEL. The  ECSEL  JU is a tri-partite 
public-private  partnership  (PPP)  jointly  funded  by  industry,  research  organisations,  participating 
Member  States  and  the  European  Commission  (representing  the  EU).    The  interim  evaluation  of 
ECSEL JU focuses on the following main aspects: 

  Effectiveness: progress made towards meeting the objectives set 
  Efficiency: extent to which ECSEL was managed and operated efficiently 
  Research  Quality:  extent  to  which  ECSEL  has  enabled  world-class  research  that  helped 
Europe  to  establish  a  leadership  position  globally,  and  how  it  engaged  with  a  wider 
constituency to open the research to the broader society  

  Openness  and  Transparency:  extent  to  which  the  ECSEL  JU  keeps  an  open  non-
discriminatory attitude towards a wide community of stakeholders and provides them with 
easy and effective access to information on the calls 

1.1.1 Importance of Electronics Components and Systems Industries 

Cyber-Physical  Systems  are  a  pervasive  enabling  technology,  which  are  impacting  all  industrial 
sectors and almost all aspects of society. Emerging industrial platforms such as the Internet of Things 
(IoT), Industrial Internet [3] and Industrie 4.0 [4] are triggering a “gold rush” toward new markets and 
are  creating  societal  scale  systems,  combining  computational,  physical  and  human  components. 
Within Europe this is being driven by the Digitising European Industry initiative [5]. The aim of this is 
to establish next generation digital platforms and re-build the underlying digital supply chain. Here 
ECSEL  has  a  pivotal  role  to  play  supported  by  other  initiatives  funded  under  the  H2020  Work 
integration,  testing  and 
Programme  addressing  platform-related  projects  and 
experimentation pilots. The aim of the large-scale pilots, which are co-funded by Member States, is 
to remove cross-border obstacles that currently prevent large-scale testing and experimentation thus 
blocking the  full deployment  of technologies  into the market. This is particularly relevant for areas 
such  as  autonomous  connected  vehicles  and  connected  smart  factories.  There  are  also  aims  to 
develop  Europe-wide 
ICT 
standardisation leading to new standards.  

foster  rapid  development  of 

for  experimentation  to 

large-scale 

facilities 

At a component level there is increasing competition in the semiconductor market, particularly China 
[6]. The Ensuring Long-Term U.S. Leadership in Semiconductors report [7] by the President's Council 
of  Advisors  on  Science  and  Technology  (PCAST)  (January  2017)  states  that  the  "concerted  push  by 
China to reshape the market in its favour, using industrial policies backed by over one hundred billion 
dollars  in  government-directed  funds,  threatens  the  competitiveness  of  the  U.S.  industry  and  the 
national and global benefits it brings“. The report concludes that: "only by continuing to innovate at 
the  cutting  edge  will  the  United  States  be  able  to  mitigate  the  threat  posed  by  Chinese  industrial 
policy and strengthen the U.S. economy”. The report recommends a three pillar strategy to (i) push 
back against innovation-inhibiting Chinese industrial policy, (ii) improve the business environment for 
U.S.-based semiconductor producers, and (iii) help catalyse transformative semiconductor innovation 
over  the  next  decade.  Delivering  on  this  strategy  will  require  co-operation  among  government, 
industry, and academia. The same considerations hold true for Europe.  

6 

 
 
in  an 

integrated  way.  The  enabling  principles  of  these  functions 

Smart  Systems  Integration  (SSI)  combines  sensing,  actuation,  data  communication  and  energy 
include 
management 
nanoelectronics, micro-electromechanics, magnetism, photonics, chemistry and radiation. Pervasive 
connectivity and digitilisation has become the backbone of modern society and here SSI technologies 
will  be  exploited  in  everyday  “connected  entities”  such  as  building-integrated  sensors,  smart  door 
locks, parking meters, road weather sensors to connected cars and implanted heart monitors. Smart 
systems integration is essential to address societal challenges ahead, such as interconnected society, 
urbanisation, ageing, social inclusion and environmental factors. The automotive sector for instance 
is the EU’s number one investor in R&D and in order to maintain this leading edge there is a need for 
innovation in vehicle electronics to meet new CO2 and emission regulations. In the healthcare sector 
there will be a need for integrated devices that make prognoses, treat acute or chronic diseases, and 
improve the quality of life of patients. Innovations will be required in bionic, biomedical, bio-sensing, 
bio-energy  harvesting,  low-power  electronics  and  secure  communications.  In  the  manufacturing 
sector  there  will  be  horizontal  integration  across  multiple  value  chains  on  processes,  data  and 
companies  and  vertical  integration  among  corporate  levels,  from  the  enterprise  resource  planning 
(ERP) level down to the field level (sensor and actuators, shop floor). Here IoT for connectivity and 
SSI  technologies  will  combine  to  provide  flexibility  to  implement  changes  both  inside  and  outside 
organisations.  

1.1.2 Key Findings of Panel 

The adoption of a tri-partite funding scheme aimed to combine resources and funding from Horizon 
2020, industry, national R&D programmes and intergovernmental R&D schemes to tackle problems 
that  could  not  be  addressed  by  single  funding  sources  alone  and  create  significant  impacts.  A  key 
goal was to increase and leverage private and public investment in the Electronic Components and 
Systems sector in Europe and by so doing strengthen Europe's future growth, competitiveness and 
sustainable development. Integration of the embedded/cyber-physical systems, nanoelectronics and 
smart systems domains has the aim of providing greater coherence of R&D across Europe which is 
currently fragmented.  

Considering this goal ECSEL has been successful in increasing the private and public investment in the 
Electronics Components  and Systems sector. Six calls have been launched between 2014 and 2016 
and  39  projects  have  been  selected  for  funding:  22  Research  and  Innovation  Actions  and  17 
Innovations Actions.  

The  total  funding  levels  for  ECSEL  were  €607.1  million  in  2014,  €628.9  million  in  2015  and  €720.9 
million in 2016, for a total of €1956.8 million with industry providing 56% of this. The EU contribution 
over this period was €463.4 million and the Member State contribution was €404.6 million. In terms 
of  gearing  each  Euro  contributed  by  the  EC  has  resulted  in  4.3  Euros  of  research  and  innovation 
activity in Europe (source ECSEL Data).  

The level of funding achieved is very positive and the ECSEL JU has had success in bringing together 
the embedded/cyber-physical systems, nanoelectronics as well as smart systems areas, however, the 
three communities are still in the process of integrating into a coherent value chain. Although there 
is commonality in the domain, with common applications, there are fundamental differences in the 
communities that are represented and the proposals that are submitted are still largely sectoral. The 
Cyber-Physical Systems area is driven by applications and here the concentration is on development 
of software and hardware that can be used across domains. In the case of nanoelectronics large-scale 
pilot  projects  have  been  funded  with  a  few  key  industrial  partners  benefitting  directly  from 
development of  new manufacturing  processes.  The smart  systems area  considers  integrated micro 
and nanosystems exploited in smart systems in a range of domains including the automotive, health 
sector, manufacturing and robotics with applications being a driver in evaluation. 

7 

 
 
An analysis of the  funded project  portfolio was performed (See  Annex A8) generating estimates of 
the relative content of CPS, smart systems and nanoelectronics funding. Estimates were derived that 
indicated  that  around  15%  of  activities  are  CPS  and  software  related,  5%  of  activities  are  smart 
systems  related  and  80%  of  activities  are  nanoelectronics  related.  This  is  in  line  with  the  split  of 
funding  going  into  the  CPS  and  nanoelectronics  domains  under  ARTEMIS  and  ENIAC  before 
integration into ECSEL (see Figure A6.5). An analysis was also performed on the probability of success 
of proposals considering the different areas. As indicated in Annex A6 the average success rates are 
lower  than  average  for  CPS  driven  projects,  except  for  IAs  in  2015.  As  a  consequence  overall  less 
funding is going into the CPS area with a lower budget per project and a lower number of submitted 
proposals.  

This  needs  to  be  addressed  in  order  to  provide  better  coverage  of  the  full  value  chain  in  future. 
Already actions have been taken by the ECSEL PAB to fund more RIAs. In the 2015 Workplan the EU 
funding for IAs reduced by 17% and was increased by 25% for RIAs (ECSEL AAR 2015, Section 5.3.1, 
p15). The funding of large pilot line projects also has an impact on the ability to spend budget and a 
problem highlighted was that Member States are unable to commit allocated budget to projects that 
is  more  funding  available  for  the 
do  not  address  their  national  priorities.  Notably  there 
semiconductor  domain  in  some  countries  at  the  national  level  than  there  is  for  the  systems  area 
which  is  more  diverse.  The  introduction  of  Lighthouse  projects  that  cluster  projects  together  to 
tackle  sectoral  issues,  e.g.  smart  factories  and  automotive  applications,  is  seen  as  a  positive  move 
that presents an opportunity to address this to some extent.  

At  the  moment  ECSEL  is operating  effectively  to support  funding  from  the  EC, Member  States  and 
industry  for  the  semiconductor  industry  but  the  systems  community  has  problems  to  secure 
sufficient funding, in particular because of the funding rules and decisions of the Member States, but 
also as a result of the proposal review process. ECSEL needs to address the value chain as a whole 
and needs the support of all parties involved in order to be able to do this. As highlighted, industrial 
support for ECSEL is strong and thus it is important to address this to ensure the continued support 
of  systems  companies  and  research  organisations.  At  the  same  time  the  national  reimbursement 
rates  for  companies  and  academic  partners  in  some  countries  are  not  favourable,  inhibiting 
engagement. 

A  second  key  goal  of  ECSEL  is  to  foster  collaboration  between  all  stakeholders  such  as  industry, 
including  small  and  medium-sized  enterprises  (SMEs),  national  authorities,  academic  and  research 
centres,  by  providing  a  focus  for  research  efforts.  In  this  respect  ECSEL  has  been  very  successful, 
however,  analysis  of  the  projects  undertaken  and  their  coherence  with  other  EC  and  National 
programmes  still  shows  some  overlap  and  a  lack  of  linkage.  It  is  clear  that  ECSEL  has  made 
considerable  efforts  to  align  activities  with  other  programmes  which  have  similar  goals,  e.g. 
CATRENE,  PENTA,  ITEA3 which  are  to  be  commended.  The  EC  has  also  encouraged  engagement  of 
projects, such as in the area of Smart CPS with ECSEL and a number of joint events have been held. 
Although  synergies  clearly  exist  surveys  of  the  JU  project  participants  indicate  that  the  level  of 
interaction is not yet optimal and there is a need for continued efforts. Here there are a number of 
opportunities going forward where ECSEL can collaborate with other key projects with similar goals. 

It is acknowledged by ECSEL that there  is a need for a top-down strategy  rather than a bottom-up 
strategy to better integrate the areas represented in the JU. Although the AENEAS pilot-line projects 
under ECSEL, large application driven projects driven by the ARTEMIS-IA, and activities supported by 
EPoSS are helping to create critical mass and align activities it is clear that there is a pressing need to 
more clearly define an overarching European Strategy that can then be used to help align with other 
European, national and regional programmes. Here there are opportunities to align with the PENTA 
EUREKA initiative and also with other JUs and initiatives that are application oriented, e.g. Factories 
of the Future, Robotics, the EIP on Active and Healthy Ageing, Big data and IoT. The adoption of the 
Lighthouse  projects  within  ECSEL  which  will  cluster  activities  into  umbrella  domains,  starting  with 

8 

 
smart factories and smart mobility, is seen as an opportunity for bringing projects together and also 
to connect with the vertical integration chain. 

A key aim in Horizon 2020 has been to engage with the SME community which accounts for 99% of 
companies  across  Europe  (20.7  million),  2/3  of  European  jobs  and  85%  of  new  jobs.  A  number  of 
dedicated  programmes  have  been  set  up  to  address  SMEs  and  the  overall  SME  budget  target  for 
Horizon 2020 is 20%. Analysis of ECSEL indicates that SME participation in 2014, 2015 and 2016 was 
30%, 28% and 24% respectively, which is very good but declining. The budget allocated to SMEs was 
12% in 2014 and 2015, rising to 13.5% in 2016. It is recommended that this should be increased going 
forward. This could be done by a number of means, however, it is recommended that the allocation 
of funding to support smaller scale experiments (e.g. €50K – €100K) is considered to encourage easy 
access  to  ECSEL  and  provide  connection  mechanisms  to  larger  companies.  Also,  more  effort  is 
needed in communicating ECSEL objectives and features to the wider public, e.g. through events and 
initiatives which can attract SMEs.  

In terms of operations the ECSEL JU is unique in the adoption of a tri-partite  funding strategy. The 
benefits in terms of funding multiplication are great and this is evidenced by the resources that have 
been mobilised to address key European issues; however, it requires considerably more effort to co-
ordinate Member States which leads to a number of administration complexities. In order to justify 
funding Member States play an active role in the governance of ECSEL, defining strategy, the funding 
priorities  and  also  in  the  selection  of  projects.  Some  Member  States  demand  their  own  project 
monitoring  in  addition  to  the  monitoring  by  ECSEL  to  justify  the  funding  allocated.  There  is  thus  a 
need  to  develop  trust  to  decrease  the  extra  administrative  reporting  burden  on  project  leaders 
imposed  by  some  Member  States.  With  the  involvement  of  the  majority  of  Member  States  across 
Europe,  with  elections  changing  governments  in  a  number  of  these  every  year  and  with  national 
funding priorities changing every year there is a significant challenge to co-ordinate, reach consensus 
and guarantee long-term funding. The overall average contribution ratio over the period 2014-2016 
was 0.87 with a variance of 0.26 which is below the goal of 1:1 matching initially set. Renewed efforts 
are thus required to increase this ratio in coming years.  

The  use  of  a  Multi  Annual  Strategic  Plan  provides  a  long-term  view  and  the  Strategic  Research 
Agenda  published  by  ECSEL  provides  a  focus  for  activities  across  Europe.  Gaining  Member  State 
commitment for multi-annual funding has been possible in some cases, e.g. the Netherlands, but in 
general  has  proved  difficult  to  achieve  across  the  majority  of  Member  States.  Notably,  national 
funding  significantly  decreased  in  spite  of  ECSEL  Participating  States´  (EPS)  initial  high-level 
commitments. This is particularly true for the 2015 calls: although EPS committed to provide €162.4 
million in funding, only €121.7 million was eventually allocated according to EC data. As Participating 
States  should  allocate  funding  commensurate  to  the  Union´s  budget,  this  decrease  in  national 
funding is partly explained by the EU´s lower financial commitments: “The EU budget has become the 
limiting factor in the selection of projects. A large amount of the ECSEL Participating States budget 
went unused (25.1% in 2015). The risk that some EPS will divest from ECSEL in view of this is growing. 
This can be partly explained by the lower committed EU budget” (ECSEL Annual Activity Report 2015, 
p.18). Thus work is needed at the highest political level to provide stability in funding as well as  to 
guarantee the continued participation of all relevant Member States. 

Complementary  measures  are  also  needed  to  ensure  funding  is  allocated  more  evenly  over  the 
Electronic  Components  and  Systems  domain  reflecting  the  general  levels  of  industry  investment 
across  the  sector.  The  funding  levels  for  companies  also  need  to  be  raised  in  order  to  make  the 
submission of proposals into ECSEL more attractive.  

A  key  justification  for  the  existence  of  ECSEL  and  the  funding  committed  to  it  by  the  EC,  Member 
States and Industry is the impact of the projects. One would expect that the project portfolio would 
be assessed by the JU itself against the top-level objectives defined in the MASRIA but this has not 

9 

 
been  done.  Within  ECSEL  this  has  been  done  separately  by  the  ARTEMIS-IA  and  AENEAS  for  the 
embedded  systems  and  nanoelectronics  areas  respectively,  with  different  approaches  to  impact 
assessment.  

The  ARTEMIS-IA  has  addressed  impact  in  a  more  rigorous  manner  by  setting  up  a  Working  Group 
specifically  to  address  monitoring  of  KPIs.  Targets  were  also  set  such  that  projects  deliver  cross-
domain  re-use  and  interoperability  for  different  product  categories  and  application  domains,  or 
promise a reduction of system design costs and development cycles for both hardware and software. 
Many projects expect commercial impact within a 3 to 5 years’ time. A key aim is to address system 
and  software  design  tools  and  environments  and  general  purpose  architectures  that  facilitate 
integration  and  reuse  across  sectors.  These  are  important  in  the  automotive,  aerospace,  factory 
automation,  industrial  processes,  smart  buildings,  energy  production  and  medical/healthcare 
sectors.  Shorter  development  cycles  and  increased  interoperability  are  important  to  achieve  a 
shorter time-to-market for new products and services in the CPS domain and the internet of things 
market. According to a survey of projects the work performed had a significant and pivotal business 
impact on reduced development costs, reduced time-to-market and in developing a new generation 
of products. Results have also been exploited in several downstream sectors.  

Looking  back  over  the  previous  ARTEMIS  JU  and  the  transition  into  ECSEL  a  number  of  notable 
success stories could be cited such as the AUTOSAR standard for automotive which was supported by 
activities performed. This is now used world-wide. The EMC2 (mixed criticality applications), CRYSTAL 
(design  environment 
integration)  and  ARROWHEAD  (cooperative  automation)  projects  have 
addressed  pan-European  industry  issues  developing  frameworks  that  can  be  shared  between 
different  communities,  platforms for interoperability and tools and methods to cope with the ever 
increasing complexity of smart digital systems. Exploitation of outputs, company growth and world 
leadership  can  be  seen  in  examples  such  as  TTTech  that  provides  core  safety-critical  data  bus 
technologies to Airbus, Boeing and NASA.   

In the case of AENEAS there is no formal follow up of impact metrics. Qualitative examples of notable 
impact were, however, available such as AMS that has used ENIAC and ECSEL funding to transform 
itself  from  a  foundry  with  commodity  products  into  a  specialist  for  producing  sensors  and  sensor 
systems, Infineon Technologies  Austria  which is the world leader in power electronic discretes  and 
modules with development of a 300 mm fab., ST Microelectronics that has a strong market position 
in  piezoMEMS  and deep sub-micron pilot fabrication capabilities, and  ASML  which  is  now  a  world 
leader in lithography that sells equipment around the world.  

The experts note that while great care is taken on measuring and reporting on input parameters such 
as funding levels, participation rates, etc., and interim measures such as detailed progress reports are 
recorded, there  is  little  objective  measurement  of  outputs.  In order  to  provide  justification  for  the 
EC, Member States and Industry to contribute to future JUs it is important to address this issue and 
follow up projects after completion to gather concrete and quantifiable evidence of impact. 

In  order  to  consider  the  impact  of  ECSEL  the  panel  performed  an  analysis  (See  Section  6)  of  the 
following criteria:  

Inclusion of SMEs 

  Engagement with the community 
 
  Number of patents and innovation outputs 
  Success in attracting public/private funding 
  Number of projects initiated, success stories and evidence of impact from projects 
  Working Groups established to support the Community 

10 

 
 
This identified a number of key successes which highlight the added value of the ECSEL. Notably over 
1000 organisations participated in ECSEL over the period 2014-2016. An analysis of the participants 
to  ECSEL  indicates  that  it  is  engaging  with  the  best  European  players  in  the  semiconductor  and 
systems domains with many key companies across a number of application sectors actively working 
on projects. A key attribute of ECSEL is it’s openness. ECSEL has strived hard to be open and anyone 
can participate in ECSEL projects. Participants do not need to be a member of ARTEMIS-IA, AENEAS 
or EPoSS. The Brokerage events are widely disseminated and new participants and project ideas are 
welcomed. If a project is funded via ECSEL the new participants are encouraged to pay a fee which is 
relative  to  their  involvement  in  the  project  to  cover  administration  costs  of  the  industrial 
associations; however, this is not mandatory which further removes barriers to engagement.  

1.2 Recommendations 

In  this  section  a  number  of  recommendations  for  the  future  are  made.  These  recommendations 
apply directly to the ECSEL JU and some recommendations also have relevance across JUs.  

Strategic Recommendations 

Recommendation 1 

Develop overall ECS 
strategy 

Action: JU with EC 
and Industry CEOs 

Although  the  MASP  and  Annual  Work  plans  set  the  short  term  goals  of 
ECSEL  well,  there  is  a  need  to  clearly  define  a  long-term,  top-down 
research, development and innovation strategy in Electronic Components 
and Systems (ECS)  to co-ordinate activities within the  ECSEL JU while still 
leaving  room  for  bottom-up  initiatives.  The  overall  strategy  should 
concentrate  on  the  integration  of  different  value  chains  (equipment, 
semiconductor,  system  integration,  photonics,  software)  and  large-scale 
pilot experiments driven by real needs in key application domains such as 
automotive, health, energy, etc. CEO level industry leaders, helped by the 
European Commission and ECSEL JU as facilitators, should create a "Digital 
Leaders Group" (on the model of the ELG – Electronics Leaders Group), to 
develop  a  strategy  for  digital  industry  in Europe. There  should  be  a  clear 
link with activities in the (DEI) Digitising European Industry initiative where 
CEOs of application domain companies are already engaged. It is important 
to understand the concerns of CEOs, provide information in their language 
and develop an accessible value proposition for technology. There is also a 
need  to  provide  a  forum  at  CEO  level  for  cross-company,  cross-domain 
priorities. In the next phase ECSEL needs to have greater levels of vertical 
integration  to  fulfil  its  mission  to  cover  the  full  ECS  domain  and  avoid 
dominance  by  single  technology  areas.  In  particular,  there  needs  to  be  a 
clear value proposition for all pilot line activities that is driven from vertical 
Lighthouse projects.  

These issues should be addressed in the proposal evaluation and selection 
criteria to improve the match of the project portfolio to strategic European 
aims.  This  is  to  ensure  optimum  coverage  of  key  areas  defined  in  the 
overarching EU ECS strategy.  

Recommendation 2 

Closer community 

It  is  still  early  days  with  respect  to  the  integration  of  activities  within 
ECSEL. In the future there needs to be greater integration of the AENEAS, 
ARTEMIS-IA  and  EPoSS  communities,  that  each  provide  different 
perspectives  but  overlap  on  technologies,  to  focus  on  integrated  vertical 

11 

 
 
integration 

Action: IAs 

roadmaps  so  that  the  European  benefits  of  investing  are  realised.  The 
three organisations have a leading position in their respective domains and 
each produce valuable Strategic Research Agendas. However, considering 
the  whole  community  these  should  be  better  integrated  into  a  single 
roadmap. 

Recommendation 3 

Emphasise vertical 
integration 

Action: JU 

Recommendation 4 

Harmonise Member 
State participation 
rules 

Action: EU with 
Member States 

Recommendation 5 

Increase SME and 
start-up 
involvement 

Action: JU 

For  selection  of  proposals  to  be  funded  the  vertical  integration  aspect 
should  be  a  dominant  evaluation  factor  to  promote  greater  cross-value 
chain  co-operation.  This  should  be  supported  by  providing  funding  rates 
that are related to TRL levels to distinguish between pilot line activities and 
systems-level  integration.  In  particular,  there  should  be  TRL-aligned 
funding  rates  in  Lighthouse  projects.  Moreover,  it  is  important  that 
proposals include measurable target results, so that the improvement over 
the  state-  of-the-art  can  be  evaluated  and  compared  across  different 
technological areas like nanoelectronics, CPS and smart systems. 

At  the  highest  political  level  a  discussion  should  be  initiated  towards 
harmonisation  and  synchronisation  of  the  Member  State  participation 
rules,  funding  rates  and  procedures  wherever  possible,  adopting  best 
practice  as the  guiding principle.  Also at the highest political level efforts 
should be made to encourage Member States to commit to a multi-annual 
funding  system  to  provide  stability  and  a 
longer-term  focus.  This 
recommendation  was  made  at  the  2nd  Interim  Review  of  the  ENIAC  and 
ARTEMIS tri-partite PPPs. The issues have proved to be difficult to address 
and  still  remain  a  barrier.  Thus  the  recommendation  has  been 
strengthened  to  emphasise  the  need  for  very  high-level  engagement  for 
resolution.  

In terms of numbers the engagement with SMEs is very positive, although 
the  percentage  of  SME  participants  in  projects  has  fallen  from  30%  in 
2014, to 24% in 2016. The budget allocated to SMEs has remained steady 
with 12% being allocated in 2014 and 2015, rising to 13.5% in 2016. Given 
the  importance  of  SMEs  the  three  Industrial  Associations  and  JU  should 
play  a more  active  role  in facilitating  the  engagement  with  SMEs.  The  JU 
should put in place a strategy to improve SME engagement and the level of 
funding being allocated to SMEs. The success of this should be monitored. 
This could be done by a number of means such as the allocation of funding 
to support smaller scale experiments (e.g. €50K - €100K) to encourage easy 
access to ECSEL and provide connection mechanisms to larger companies. 
A plan for supporting start-ups should be considered with actions to align 
national  and  regional  investment  in  Europe  for  innovation  such  as  the 
ARTEMIS Centres of Innovation Excellence. 

Recommendation 6 

Engage with CEOs 
across the Systems 
Community 

Action: JU 

in  Recommendation  1 

As  highlighted 
in  order  to  define  a  global 
overarching  strategy  there  is  a  need  for  CEO  engagement.  Going  beyond 
this  there  is  a  need  to  better  engage  the  CEOs  of  systems  companies  to 
encourage participation in ECSEL addressing key issues, not only strategy, 
but  also  funding  levels,  project  types  and  success  rates  in  order  to make 
ECSEL an attractive proposition for industry and ensure the achievement of 
innovation  across  complete  vertical  value  chains  targeting  European 
needs. 

12 

 
Recommendation 7 

Explore synergies 
with other JUs and 
LSPs 

The  JU  should  explore  synergies  with  the  large-scale  pilots  and  other 
relevant JUs to maximise the benefits of work and ensure coherence with 
other  initiatives.  Here  the  working  groups  set  up  by  EPoSS  addressing 
sectors  such  as  automotive,  healthy  living,  manufacturing  and  robotics 
may have a role to play. 

Action: JU 

Operational Recommendations 

Recommendation 8 

Promote ECS 
Strategy 

Action: JU 

Based  on  an  overall  top-down  Electronic  Components  and  Systems  (ECS) 
strategy  targeted  calls  should  be  made  to  address  sectors  and Brokerage 
Events  should  be  used  to  promote  the  sectoral  integration  of  projects 
around  topics  of  importance  to  Europe,  in  which  European  companies 
have  a  chance  to  remain/become  leading  global  players  or  gain  market 
share. 

Recommendation 9 

Monitor impact post 
project 

Action: JU 

Recommendation 10 

Benchmark 
internationally 

Action: JU 

Although  it  is  clear that  ECSEL  outcomes  are creating  impact,  in order  to 
more  accurately  measure  impact  and  success  of  projects  appropriate 
metrics  should  be  collected  during  project  execution  and  also 
subsequently  after  a  project  is  completed  (e.g.  12  months  after  project 
end). Although it is difficult to give quantitative metrics for the impact of 
complex  projects  there  is  a  need  to  gather  appropriate  metrics  from 
projects to support an assessment of impact against the goals of the JU. It 
is recommended that some budget is reserved for post project evaluation 
and this should be mandatory for future projects funded under the ECSEL 
JU. This recommendation was made previously at the 2nd Interim Review of 
ENIAC  and  ARTEMIS.  ECSEL  representatives  highlighted  that  in  practice 
that  it  had  not  been  possible  to  follow  this  up  in  ECSEL  due  to  lack  of 
resource.  Thus  the  need  for  budget  allocation  to  support  this  and  the 
mandatory nature of the review is highlighted.  

In  order  to  benchmark  at  an  international  level  it  is  important  to  also 
provide  assessment  from  international  experts.  It  is  recommended  that 
reviewers  from  other  geographies  such  as  Asia  and  the  US  should  be 
considered  during  project  selections  with  a  view  to  ensuring  that  the 
projects  being selected are  truly world class. If such is the case, then the 
project  goals will deliver impacts which will enable those  stakeholders to 
win on the global stage. Care needs to be taken, however, in selecting the 
reviewers  so  as  to  avoid  any  conflicts  of  interest  which  might  result  in 
competing  international  industries  gaining  insights  into  European  ideas. 
This  could  be  achieved  by  allowing  stakeholders  a  veto  on  international 
reviewer selections. 

Recommendation 11 

Trace reuse of 
project results 

Action: JU 

For  traceability  of  impact  and  to  show  how  projects  build  upon  previous 
results  it  is  important  to  maintain  a  log  of  how  project  results  are  used 
from project to project. This has been done to a certain extent but it would 
be  useful  to  also  indicate  the  "share  of  reuse",  i.e.  an  assessment  of 
funding  reuse  with  respect  to  the  full  budget.  Additionally,  this  should 
apply  to  reuse  both  into  and  from  other  H2020  programmes.  This  will 
become  more  important  as  Lighthouse  projects  are  initiated  via  ECSEL 

13 

 
 
which  will  cluster  activities 
funded  by  different  sources.  This 
recommendation  was  made  at  the  2nd  Interim  Review  of  ENIAC  and 
ARTEMIS  but  is  re-iterated  as  it  will  become  more  important  as  ECSEL 
engages in more collaborative activities. 

Recommendation 12 

Reduce 
administration and 
concentrate on 
strategy 

Action: JU 

The  administrative  burden  should  be  reduced  so  that  the  JU  Governing 
Boards  can  spend  more  time  addressing  strategic  issues.  This  was 
recommended  at  the  2nd  Interim  Review  of  ENIAC  and  ARTEMIS  and 
interviews  of  ECSEL  board  members  highlighted  that  progress  had  been 
made  to  reduce  overhead.  However,  this  was  still  considered  to  be  a 
barrier  and  although  AENEAS,  ARTEMIS-IA  and  EPoSS  have  very  good 
industrial representation from key companies, higher level involvement is 
encouraged. 

Recommendation 13 

Reduce 
management 
overhead for 
participants 

Action: JU 

Recommendation 14 

Streamline review 
and reporting 
processes 

Action: JU, Member 
States 

Specific  support  mechanisms  should  be  developed  to  enhance  project 
management processes. A key recommendation is that management costs 
should be 100% funded by the EC for all JTI projects. This recommendation 
was  made  in  the  2nd  Interim  Review  of  ENIAC  and  ARTEMIS  but  was  not 
taken  up  and  has  not  been  implemented  in  ECSEL.  Survey  respondents 
highlighted  that  management  overhead  of  engaging  with  ECSEL  was  a 
barrier to engagement, so it is still considered to be an issue. Additionally, 
it is recommended this time that lessons learned should be collected from 
projects with the aim of improving management effectiveness by analysing 
project communication process models, tools and practises among project 
participants,  identifying  best  practices  and  introducing  them  into  the 
project  management  schemes  to  streamline  processes.  The  outcomes  of 
this work could also be exploited by other JUs. 

Projects  should  be  subject  to  a  unique  review  and  reporting  process  to 
avoid an unnecessary overhead. Efforts should be made to harmonise the 
financial  reporting  process,  remove  duplication  and  encourage  Member 
States to develop trust in the level of reporting provided to the JU. A level 
of trust may be partially achieved via improving metrics collection and via 
impact  assessment  of  projects.  This  recommendation  was  made  at  both 
the  1st  and  2nd  Interim  Reviews  of  ENIAC  and  ARTEMIS.  This  has  been 
addressed but it is acknowledged that this is a complex issue and it has yet 
to  be  resolved.  Notably  survey  respondents  highlight  this  as  being  a  key 
issue  which  merits  the  recommendation  to  still  consider  ways  of 
simplifying procedures. 

Recommendation 15 

Harmonise re-
imbursement rates 

Action: Member 
States 

There is a need to harmonise re-imbursement rates at a national level as 
the  current  system  makes  engagement  of  actors  from  some  countries  in 
ECSEL  unattractive.  Much  has  already  been  achieved  by  the  Public 
Authorities Board (PAB) in bringing the  Member  States  together, but  this 
issue  remains  largely  unresolved.  Here  there  needs  to  be  further  action 
from Member States. 

14 

 
 
 
 
1.3 Observations 

As well as the previously described recommendations, a number of observations were also made, the 
learnings from which could be applied to other JU programs. 

Observations 

Observation 1  A  key  success  of  ECSEL  has  been  in  attracting  gearing  funding.  This  directly 
supports the key goal of ECSEL which is to attract private and public investment 
in the Electronics Components and Systems sector.  In particular there has been 
very strong support from industry (56% of funding is industrial). This has allowed 
39 projects to be supported with €1.956.8 billion of funding in the period 2014-
2016 including the contributions from the EC and MS. This level of funding would 
not  have  been  available  without  the  adoption  of  the  tri-partite  approach. 
Notably ECSEL highlights that each Euro contributed by the EC has resulted in 4.3 
Euros of research and innovation activity. Key to generating this level of interest 
and support has been the projects that have been supported. These address key 
technological  developments  in  the  semiconductor  domain  and  also  major 
societal challenges, such as affordable healthcare and well-being, green and safe 
transportation,  Smart  Cities,  etc.,  which  are  strategically  key  for  European 
companies  unlocking  investment.  Here  there  is  a  need  to  encourage  more 
activity in the CPS domain to engage  at the  applications level. These areas  also 
present new business opportunities for SMEs and start-ups. It is notable that in 
order  to  ensure  that  the  level  of  industrial  investment  is  maintained,  and 
potentially increased, projects should be driven by industrially relevant research 
which is strategically linked to industrial roadmaps. 

To support and  encourage investment by industry the Member State funding is 
very  important.  Different  national  priorities  and  the  flexibility  in  the  rules  for 
national Member State co-funding introduced in ECSEL has resulted in significant 
variations of co-funding resulting in an average level of 0.87 in the period 2014-
2016 with a variance of 0.26. Ideally countries should be encouraged to commit 
to at least 1:1 funding. 

Observation 2  A  key  aim  of  the  ECSEL  JU  is  to  bring  together  the  fragmented  Electronic 
Components and Systems community to tackle  areas that will lead to a greater 
impact.  The  stakeholder  participation  in  ECSEL  has  been  very  good  with  over 
1000 organisations participating in ECSEL over the period 2014-2016. There has 
also  been  a  good  balance  of  participation  in  terms  of  numbers  between  large 
industry  (37%),  SMEs  (27%)  and  Higher  Education  Institutions  and  Research 
Organisations (36%). Partly this can be attributed to the mixture of high and low 
TRL  projects  undertaken.  Responses  to  a  survey  of  applicants  indicated  that 
cross-border  collaboration  on  new  research  is  a  key  driver  with  the  aim  to 
increase  knowledge  and  experience  working  with  new  company  partners. 
Surveys  also  indicated  that  ECSEL  was  creating  synergies  between  components 
and  systems  and  along  the  value  chain  which  allowed  companies  to  better 
address  market  needs. However,  as highlighted in Recommendation 1 the level 
of  vertical  integration  is  still  not  sufficient.  It  is  noted  that  it  is  important  for 
ECSEL to perform a mixture of high and low TRL projects that produce both new 
knowledge  as well as new products, but  also with the  aim of better integrating 

15 

 
 
the  European  value  chain  as  this  will  strengthen  Europe’s  position  as  an 
Electronics Components and Systems supplier. 

Observation 3  At the time of this interim evaluation it is too early to assess the  impact of the 
still  on-going  and  newly  funded  projects.  ECSEL  has,  however,  been  active  and 
successful in promoting the outcomes of the JU. The ECSEL Impact Document and 
Book of Projects which have  been produced, are both very good for promoting 
the  benefits of ECSEL to industry and potential participants. Notably ECSEL also 
performed  an  ECSEL  JU  Impact  Analysis  Study  in  2016.  Here  it  is  reported  that 
75% of participants rated the impact of participating in the JU as highly positive 
and beneficial to their organisation. This indicated a number of drivers for impact 
that  were  not  necessarily  market  or  product  related  such  as  technology 
development,  benchmarking,  networking  on  a  global  level,  and  gaining  insights 
into  future  developments.  The  impact  analysis  study  should  be  continued  and 
performed on a regular basis, also considering the wider impacts for participants, 
particularly  considering  how  ECSEL  supports  the  Electronic  Components  and 
Systems community. 

Observation 4 

together  embedded  systems/cyber-physical  systems 

The  intended  role  and  objective  of  ECSEL  is  to  keep  Europe  at  the  forefront  of 
technology  development  in  the  area  of  Electronic  Components  and  Systems 
bringing 
(ARTEMIS), 
nanoelectronics (ENIAC) and smart systems integration (EPoSS). The combination 
of  the  three  domains  supported  by  private  and  public  investment  has  allowed 
problems  that  could  not  be  addressed  by  single  funding  sources  alone  to  be 
tackled to create significant impacts. Analysis of ECSEL participants indicates that 
the  Best  European  players  from  leading  semiconductor manufacturers,  systems 
and  smart  integration  companies  are  active  in  projects  and  that  some  of  these 
companies  are  engaged  in  multiple  projects.  In  order  to  ensure  that  ECSEL 
remains  relevant  for  industry  and  meets  the  needs  of  key  European  player’s 
regular  surveys  should  be  performed  to  ensure  that  the  experiences  and 
concerns  of  these  companies  are  recorded  along  with  any  barriers  to 
participation. 

Observation 5  With respect to openness it was very notable in a survey of ECSEL applicants that 
a  large  share  of  respondents  (191,  43%)  had  submitted  a  proposal  for  the  very 
first time to ECSEL with no previous experience from other JUs. Respondents also 
highlighted  that  an  attraction  was  that  the  programme  provided  stronger 
support  from  idea  to market.  ECSEL  has  thus  been  successful  in  attracting  new 
players  via  their  brokerage  events.  New  stakeholders  who  attend  brokerage 
events should be asked to highlight the reason why ECSEL is attractive to them to 
better understand how ECSEL can meet their needs in future and engage better 
with the community. 

Observation 6 

The  MASP  and  Annual  Work  Programme  have  been  successful  in  defining  the 
research direction and call priorities of ECSEL. It was noted that it is challenging 
to  set  the  research  agenda  as  there  is  a  push  from  the  public  authorities  for  a 
narrower  and  more  focussed  research  agenda,  whereas  private  companies, 
would like a broader approach that can better respond flexibly to market needs. 
The  balance  is  currently  leading  to  commitment  of  56%  funding  from  industry 
which is very positive but there are concerns that smaller private members, e.g. 
SMEs,  do  not  get  a  voice  in  the  drafting  of  the  MASP  or  the  Annual  Work 
Programme.  Partly  this  is  due  to  the  fact  that  they  do  not  have  the  means  or 

16 

 
resources  to  attend  all  the  meetings.  Here  there  needs  to  be  a  means  of 
collecting the views of SMEs to ensure that the whole community is included. 

Observation 7  A major new initiative, Lighthouse projects were launched in the 2016 Work Plan. 
This is seen as a very positive move with the aim of improving and accelerating 
the impact of the ECSEL JU by engaging actors across the supply/value chain to 
achieve concrete socio-economic objectives. The concentration on platforms for 
“Smart  X”  markets  is  likely  to  have  an  impact  on  integrating  the  electronic 
components  and  systems  community  by  creating  ecosystems  across  value  and 
supply  chains.  This  new  instrument  has  great  promise  and  it  is  recommended 
that it is closely monitored in order to identify issues that may arise at an early 
stage  so  that  corrective  actions  can  be  taken  as  required.  For  these  large 
collaborative  projects,  there  should  be  sufficient  funding  for  management, 
including  activities  to  ensure  cohesion  between  the  project  parts.  The  global 
aspect  to  this  work  should  be  considered  to  ensure  that  the  outcomes  are 
competitive on a world-class level so that the aim to strengthen Europe's future 
growth, competitiveness and sustainable development are met. 

Observation 8 

The overall success rates for IA projects has been 43-45% over the 3 calls made 
between 2014 and 2016. The success rates for RIA projects dipped from 18% in 
2014 to  16%  in  2015.  However  corrective  actions were  performed  by  the  JU  in 
2016 and the success rate for RIA projects increased to 29%.  

The  trend  in  success  rates  for  CPS  related  RIA  projects,  however,  shows  a 
reduction  in  success  over  the  past  two  years  with  the  2016  success  rate  being 
23%  compared  to  the  global  29%  success  rate  for  proposals  in  ECSEL.  Likewise 
the  success rate for CPS IA projects in 2016 was 32%  compared with an overall 
success rate for ECSEL proposals of 46%. This appears to be due to a combination 
of  factors  including  lower  average  budget  for  CPS  projects  and  also  a  lower 
number of proposals being submitted.  

Survey  results  indicated  that  in  general  proposers  thought  that  the  cost  of 
submitting a proposal and the success rate of ECSEL proposals were reasonable. 
Respondents also generally considered that the benefits of participation in ECSEL 
proposals was  worth  the  investment.  However, with  respect  to  success  rates  it 
was  notable  that  Research  Organisations  and  SMEs  tended  be  less  favourable 
and a small number of respondents also indicated that the benefits of submitting 
an ECSEL proposal was lower than their investment. As the aim is to be as open 
as possible, the JU should find ways to make participation of SMEs and ROs more 
attractive. Measures include involving them more strongly in the formulation of 
the  MASP  and  annual  work  plans  and  finding  ways  to  connect  them  more 
strongly in the value chain. 

In  a  survey  of  participants  the  new  procedures  introduced  under  H2020  within 
ECSEL  were  seen  to  be  having  an  impact  on  the  lifecycle,  reducing  the  time 
needed to sign contract, for payment and for first project results. Proposers were 
also generally satisfied with the proposal process. The only criticisms came from 
the  proposal  evaluation  and  selection  process  which  was  thought  to  lack 
transparency.  The  move  to  very  large  projects  has  benefits  and  pitfalls.  The 
benefits are that major industrial issues can be tackled. The negatives are that in 
the selection process  a large  proposal is more likely to have  a mix of good and 
bad parts. This makes proposal evaluation more dependent on the background of 
the evaluator and how  he/she sees and places emphasis on these. Here efforts 

17 

Observation 9 

 
Observation 
10 

should be made to improve the transparency of the evaluation process and also 
ensure that proposals are evaluated and ranked appropriately. 

The  needs  and  community  drivers  of  the  production  oriented  semiconductor 
industry  and  the  agile  ecosystem  oriented  embedded  SW  and  smart  systems 
integration communities are very different. This makes it particularly difficult for 
evaluators to compare impact. For the semiconductor industry this is calculated 
via  turnover  of  mass  production.  The  impact  of  embedded  SW  is  much  more 
difficult to evaluate, as it is the core of nearly all smart systems such as vehicles, 
smart  appliances,  etc.  Notably  the  software  alone  has  no  value.  Similarly  the 
impact  of  smart  systems  integration  is  also  difficult  to  evaluate.  The  overall 
system  consists  of  mechanics,  hardware  and  software  and  it  is  impossible  to 
separate  the  influence  of  the  different  parts.  Therefore  there  is  a  need  to 
consider  how  the  evaluation  of  proposals  in  the  different  domains  can  be 
changed to provide a more level playing field. 

18 

 
 
 
 
 
European Commission 

Interim Evaluation of the ECSEL Joint Undertaking (2014-2016) Operating under Horizon 
2020 – Executive Summary 
Luxembourg, Publications Office of the European Union 

2017 – 20 pages 

ISBN 978-92-79-69626-8 

doi:10.2759/517786 

19 

 
 
 
 
 
 
 
K
K
-
0
1
-
1
7
-
6
1
7
-
E
N
-
N

     doi:10.2759/517786            ISBN 978-92-79-69626-8 

20