Reply of the Commission to the Special Report "Is the design and management of the mobility scheme of the Leonardo da Vinci programme likely to lead to effective results?"
Date: 2010-08-06
Institutions: European Commission
Subjects: EU financing, administration of the Institutions, management accounting, project evaluation, report, vocational training

EN 

EN 

   EN 

 
EUROPEAN COMMISSION 

Brussels, 6.8.2010 
SEC(2010) 983 final 

Reply of the Commission to the Special Report  

"Is the design and management of the mobility scheme of the Leonardo da Vinci 
programme likely to lead to effective results?"

EN 

1 

   EN 

 
Reply of the Commission to the Special Report  

"Is the design and management of the mobility scheme of the Leonardo da Vinci 
programme likely to lead to effective results?" 

EXECUTIVE SUMMARY 

I-III. The Commission welcomes the overall conclusion of the Court of Auditor's report that 
the  design  and  management  of  the  mobility  scheme  of  the  Leonardo  da  Vinci  programme 
could lead to effective results.  

Indeed, the programme shows increasing demand which at present can be only partly covered 
(less  than  50  %  of  applications  can  be  awarded  within  the  available  budget).  More  than 
80.000  people  were  financed  in  2009  (67  %  persons  in  initial  training,  20  %  people  on  the 
labour  market,  13  %  teachers/trainers).  By  far  the  majority  of  participants  (nearly  90  %) 
expressed themselves as highly satisfied with the stay abroad in their final reports.  

The  positive  impact  on  participants  in  the  previous  programme  has  been  confirmed  by  the 
impact  study  published  in  2007.  The  mobility  experiences  improved  the  skills  and 
competences of young people and positively influenced their employability, as well as their 
personal  development.  A  study  on  the  impact  of  the  Leonardo  mobility  of  teachers  and 
trainers  in  vocational  education  (VETPRO)  is  about  to  be  finalised,  and  the  initial  results 
confirm the positive effects not only on the personal development of participants but also on 
the Vocational Education and Training schools, training providers and systems.  

The  Commission  welcomes  the  Court's  acknowledgement  that  an  appropriate  project  life-
cycle  management  system  has  been  established  and  that  the  operational  aspects  of  the 
programme were generally well managed. The Commission underlines the fact that many of 
the  management  recommendations  are  valid  for  the  whole  Lifelong  Learning  Programme 
(LLP), because the Leonardo Programme forms an integral part of the LLP. 

IV. The Commission stresses that the audit mainly covered the years 2007-2008, which were 
the  first  two  years  of  a  new  integrated  programme.  Furthermore,  the  programming  and 
reporting  cycle  with  National  Agencies  covers  two  years  from  the  National  Agency’s 
preparation of the draft work programme to the finalisation of the yearly report assessment by 
the  Commission.  Therefore,  improvements  made  afterwards,  notably  in  2010,  have  not  all 
been taken into account by the Court. 

In addition, it has to be noted that results of actions co-financed in 2007-2008 are often not 
visible immediately after the actions, as mobility and training normally take several years to 
show their effects.  

IV. first indent 

For  the  design  of  the  current  Leonardo  da  Vinci  programme,  the  Commission  launched  a 
consultation  process  with  National  Agencies,  National  Authorities  and  stakeholders  to  take 
into  account  earlier  experiences.  The  Commission  took  into  consideration  not  only  the 
findings and suggestions from the mandatory evaluations, but also used other external studies 
as a source of ideas and to provide additional insight.  

EN 

2 

   EN 

IV. sixth indent 

The  Commission  is  aware  that  the  quality  of  secondary  controls  by  National  Authorities  on 
National Agencies varies, as was also pointed out by the Court in its Statement of assurance 
for  the  financial  year  2008.  Following  this  earlier  observation  from  the  Court,  the 
Commission  has  taken  action  and  developed  detailed  guidelines  for  National  Authorities  in 
order to provide both methodological support and practical tools for the secondary controls by 
the  National  Authorities.  These  guidelines  were  issued  to  the  Member  States  in  December 
2009 and were also the main subject of a seminar for National Authorities held in March 2010 
in order to remedy the weaknesses found. 

IV. seventh indent 

The Commission accepts that the work on performance indicators has to be finalised.  

It underlines, however, that the work with the programme committee on the establishment of 
indicators started already in 2007. Furthermore, based on the impact studies and evaluations 
of  the  previous  Leonardo  programme  and  the  available  quantitative  data  and  the  qualitative 
feedback  from  participants  in  the  current  programme,  the  Commission  is  confident  that  the 
Leonardo da Vinci mobility programme is producing effective results.  

V.  The  Commission  accepts  the  recommendations  made  by  the  Court  and  has  already 
implemented  them  in  full  or  partially,  including:  the  qualitative  feedback  to  National 
Authorities  on  the  yearly  National  Agency  reports,  the  integration  of  the  yearly  activity 
reports into the monitoring visits, the alignment of the annual activity report structure with the 
work  programmes,  the  inclusion  of  results  oriented  indicators  in  National  Agencies'  work 
programmes and the finalisation of LLPLink. 

The Commission wishes to underline that: 

–  It  has  already  vastly  improved  the  supporting  IT  system  for  the  management  of  the 

programme in 2009 and is further improving its reporting facilities in 2010; 

–  It  already  provides  qualitative  feedback  on  the  yearly  work  programmes  of  the  National 
Agencies,  but  for  effectiveness  reasons  it  has  limited  its  feedback  to  those  cases  that 
require revision because they do not correspond to the expected quality standards; 

–  It shares the Court’s view that support in finding partners is important and it is exploring 
other  options,  since  the  traditional  partner  finding  database  did  not  deliver  the  quality 
service expected; 

–  It  has  already  developed  detailed  guidelines  for  assessing  project  applications.  The 
Commission considers that the supervision of the quality assessment of applications is part 
of  the  remit  of  the  National  Authorities’  secondary  controls  of  National  Agencies; 
nevertheless, it is testing whether it can monitor these procedures more closely, taking due 
account of cost-efficiency; 

–  It  included  performance  indicators  in  the  setup  for  the  2011  National  Agency  work 
programmes, which were welcomed by the National Agency directors at their meeting of 
20 May 2010 and will therefore be applied as of the planning and reporting period 2011; 

EN 

3 

   EN 

–  The Commission accepts that the work on performance indicators needs to be finalised. It 
underlines, however, that the work with the programme committee on the establishment of 
indicators  started  already  in  2007.  The  Commission's  proposal  was  adopted  by  the 
committee at its meeting of 17 June 2010. On this basis, the relevant data will be matched 
with the  now established  indicators; the  first  results  of this exercise are expected by mid 
2011. Furthermore, based on the impact studies and evaluations of the previous Leonardo 
programme  and  the  available  quantitative  data  and  the  qualitative  feedback  from 
participants in the current programme, the Commission is confident that the Leonardo da 
Vinci mobility programme is producing effective results.  

AUDIT SCOPE AND APPROACH 

13.  The  Commission  highlights  the  fact  that  2007  and  2008  were  the  first  two  years  of  the 
Lifelong Learning Programme which, for the first time, integrated Leonardo da Vinci and the 
former  Socrates  programme  into  a  single  programme.  As  a  consequence,  new  common 
procedures  had  to  be  put  in  place.  Furthermore,  the  programming  and  reporting  cycle  with 
National  Agencies  covers  two  years  from  the  preparation  of  the  work  programmes  by 
National  Agencies  to  the  finalisation  of  the  yearly  report  assessments.  Therefore, 
improvements made afterwards, notably in 2010, have not all been taken into account by the 
Court.  

OBSERVATIONS 

16.  The  Commission  underlines  the  fact  that  results  of  actions  supported  in  2007-2008  are 
often not visible immediately after the actions, as mobility and training normally take several 
years for their effects to be seen.  

18.  The  Commission  welcomes  the  Court’s  finding  that,  in  designing  the  new  Leonardo 
programme, the results of the mid-term and final evaluations of the predecessor programme 
were taken into account by means of structured action plans.  

Starting  in  2007,  the  Commission  has  been  extensively  discussing  with  the  programme 
management  committee  a  set  of  programme  impact  indicators  applicable  to  the  LLP  as  a 
whole.  The  committee  expressed  "no  opinion"  on  two  proposals  tabled  by  the  Commission 
and  asked  for  further  analysis  on  cost-effectiveness.  The  new  proposal  was  adopted  by  the 
committee  at  its  meeting  of  17  June  2010.  On  this  basis,  the  relevant  data  will  be  matched 
with the now established indicators; the first results of this exercise are expected by mid 2011.  

19-20.  The  Commission  maintains  that  it  is  not  its  role  to  follow  up  whether  the 
recommendations  set  at  national  level  have  been  followed  up  by  the  National  Authorities. 
However,  the  Commission  could  invite  National  Authorities  to  undertake  such  follow-up  at 
national level.  

The Commission emphasizes that the final national reports that were presented in due time by 
the  National  Authorities  had  been  submitted  to  the  external  contractor  for  analysis  and  the 
results  have  been  validated  by  the  Commission.  Cross-cutting  issues  have  been  included  in 
the final action plan established by the Commission.  

22. The Commission considers that external studies which are not formal evaluations of the 
programme do not require the drawing-up of action plans.  

EN 

4 

   EN 

The  content  of  external  studies  such  as  the  Study  on  obstacles  to  transnational  mobility 
"MoVE-iT"  and  the  Impact  Analysis  of  Leonardo  da  Vinci  Mobility  on  young  trainees  and 
employees, which are quoted by the Court, is used as background information and as a source 
of  ideas  for  further  development  or  adaptation  of  the  programme  and  its  rules.  They  also 
provide a more detailed insight into certain aspects of the programme actions. 

Box 1 

First indent 

The Commission agrees that there is still room for improvement in the implementation of the 
programme  in  the  host  countries  and  in  the  preparation  of participants as pointed out in the 
recommendation of the impact study mentioned. However, the study also concludes that the 
fundamental  principles  and  mechanisms  of  the  programme  neither  need  nor  ought  to  be 
changed.  The  study  concludes  that:  “the  Leonardo  da  Vinci  mobility  programme  can  be 
described as particularly efficient, effective and cost-effective.”  

27.  The  Commission  welcomes  the  Court’s  overall  appreciation  of  the  management  of  the 
project life cycle for the programme. 

As the Court points out, the supporting IT system could be improved. The Commission notes 
that a totally new IT system, covering a multiplicity of sub-programmes and actions had to be 
put in place for the new LLP, of which Leonardo da Vinci is only a part. In the meantime, the 
quality  of  implementation  reporting  has  increased  and  is  now  robust.  Furthermore,  8 
electronic application forms were offered for the 2010 call, two report forms are in production 
and  three  more  are  due  to  be  put  into  production  in  2010.  The  overwhelming  majority  of 
Leonardo  da  Vinci  applications  are  thus  being  treated  electronically  in  2010,  thereby 
increasing efficiency and reducing the scope for manual input errors. 

28. The Commission reiterates that, with regard to IT systems, important developments took 
place in 2009 and continued in 2010. Improvements made after the Court’s audit, notably in 
2010, have not all been taken into account by the Court. 

29.  Since  the  start  of  the  LLP,  the  Commission  continued  to  gather  quantitative  data  and 
qualitative  feedback  from  participants  on  Leonardo  da  Vinci  mobility  through  Rap4Leo1, 
resulting in an uninterrupted and reliable data series for the programme.  

In addition, the Commission underlines that, since the end of 2008, LLPLink has been used 
for  the  production  of  uniform  and  standardised  financial  reports  and  management  statistics. 
The  combination  of  the  statistical  data  from  Rap4Leo  and  LLPLink  ensure  the  quality  and 
coherence of data for Leonardo da Vinci mobility.  

Starting  in  2007,  the  Commission  has  been  extensively  discussing  with  the  programme 
management  committee  a  set  of  programme  impact  indicators  applicable  to  the  LLP  as  a 
whole.  The  committee  expressed  "no  opinion"  on  two  proposals  tabled  by  the  Commission 
and  asked  for  further  analysis  on  cost-effectiveness.  The  new  proposal  was  adopted  by  the 
committee  at  its  meeting  of  17  June  2010.  On  this  basis,  the  relevant  data  will  be  matched 
with the now established indicators; the first results of this exercise are expected by mid 2011. 

1 

Rap4Leo is an IT tool that makes it possible, among other things, to collect quantitative data and the 
opinions of participants in Leonardo da Vinci mobility actions.  

EN 

5 

   EN 

                                                 
31.  The  Commission  welcomes  the  Court’s  finding  that  arrangements  for  approving  the 
National Agencies’ annual work programmes are appropriate.  

For  reasons  of  cost-effectiveness,  the  Commission  has  limited  its  feedback  to  National 
Agencies on their draft annual work programme to those cases requiring revision, in order to 
ensure that the expected quality standards are achieved. 

The  Commission  underlines,  however,  that  it  does  provide  the  National  Authorities  and 
National  Agencies  with  detailed  qualitative  feedback  on  programme  management  and 
implementation in its evaluation conclusions of the yearly National Agency report, in which it 
also indicates expected remedial action, as needed. The Commission also checks that remedial 
actions  are  included  in  the  following  National  Agency  work  programme,  to  ensure  a 
continuous improvement in programme implementation. 

32.  The  Commission  welcomes  the  Court’s  opinion  that  the  participating  countries  and  the 
Commission publicise and promote the programme satisfactorily. 

33-35. The Commission is aware of difficulties on the part of some applicants’ in finding host 
partners  in  other  countries,  and  notes  the  suggestion  to  facilitate  this  process.  The  previous 
partner search database which existed under the Leonardo da Vinci II programme was closed 
down  because  it  could  not  ensure  a  sufficient  quality  of  data  entries  into  the  system.  Other 
solutions  have  to  be  explored  as  suitable  alternatives.  A  promising  example  is  the 
EuroApprenticeship network and platform, set up by competent bodies including Chambers of 
Crafts or Commerce and Vocational Education and Training providers, which started in 2010 
following a call for proposals, and which may be better suited to address the issue of finding 
quality host enterprises. The Commission considers that such alternatives could be explored 
for the next programme generation. 

36.  The  Commission  is  pleased  to  note  that  the  procedures  for  assessing  applications  and 
selecting  projects  in  the  participating  countries  are  found  to  be  transparent,  well-organised, 
and in line with the Commission’s instructions. 

It stresses that the rules of procedure in the Guide for National Agencies are designed to give 
sufficient assurance on the quality of the selected grants. 

37.  As  established  by  the  Commission’s  updated  guidelines  issued  in  2009,  a  quality 
assurance  check  on  the  assessments  of  grant  applications  should  be  part  of  the  secondary 
controls on the National Agencies by the National Authorities. The grant award procedure - of 
which  the  assessment  of  grant  applications  is  an  important  element  -  is  part  of  the  “key 
controls”  on  which  the  National  Authorities  have  to  express  their  opinion  in  their  yearly 
declaration of assurance.  

41. A revised setup for the National Agency work programmes has been designed for 2011, 
including  targets  and  performance  indicators  which  will  further  facilitate  meaningful 
comparison of results against planned performance. The new approach was welcomed by the 
National Agency directors at their meeting of 20 May 2010 and will thus be implemented for 
planning and reporting as of 2011. 

43. In order to address the lack of consistency in the National Agencies’ approach to annual 
activity  reporting,  the  Commission  has  provided  a  single  on-line  report  form  for  use  by  all 
National  Agencies.  In  addition,  based  on  the  experience  of  previous  years,  the  Commission 
provided detailed instructions for completion of the yearly reports in 2009. 

EN 

6 

   EN 

Box 2 

The  Commission  underlines  that  the  weaknesses  found  by  the  Court  in  the  way  in  which 
National Agencies presented the results of the implementation of the 2007 work programmes 
refer to three out of the six National Agencies examined and did not systematically occurr in 
each of the three.  

44.  As  from  the  2008  annual  activity  reports  onwards,  the  Commission  has  already  sent 
extensive  evaluation  conclusions  to  the  National  Authorities  and  National  Agencies.  These 
evaluation  conclusions  provide  detailed  feedback  on  the  quality  of  the  programme 
implementation  and  management  in  the  country  concerned,  and  aim  to  guide  National 
Agencies so that they improve their performance. Thus, a direct link is established between 
the National Agency work programme and the annual activity report. 

46.  National  Agencies  are  indeed  required,  as  the  Court  notes,  to  undertake  monitoring 
activities  to  support  programme  beneficiaries,  in  addition  to  checks  of  beneficiaries  which 
focus  on  ensuring  correct  use  of  EU  funds.  In  a  context  of  limited  resources  in  National 
Agencies,  and  considering  the  substantial  control  requirements,  the  Commission  did  not 
consider  it  appropriate  to  set  minimum  requirements  for  National  Agencies  for  monitoring 
visits  of  beneficiaries.  This  allows  National  Agencies  to  organise  the  monitoring  of 
beneficiaries  in  the  most  effective  and  appropriate  way,  taking  national  contexts  and 
constraints  into  consideration,  for  example  replacing  monitoring  visits  to  individual 
beneficiaries with monitoring meetings bringing together groups of beneficiaries. 

47. The concept of "Quality and impact monitoring visits" was newly introduced in 2008 and 
has been gradually improved since then in order to gain firm assurance of the effects of the 
programme on the basis of examples of actual projects. Feedback has been sent to National 
Agencies  on  a  regular  basis  since  August  2008.  For  the  second  round  of  the  Leonardo  da 
Vinci Quality and Impact Monitoring starting in 2010, visits will have a more regional focus.  

48. The objectives of the Leonardo Programme are specified in the legal basis of the Lifelong 
Learning  Programme,  which  has  been  agreed  by  the  European  Parliament  and  the  Council. 
The Commission is working on developing suitable indicators to these objectives for the LLP 
as a whole. This is a particularly complex exercise, given the fact that 58 objectives have been 
stipulated in the legal basis. The Commission accepts that the overall system for reporting on 
the  results,  effects  and  impact  based  on  performance  indicators  has  yet  to  be  finalised. 
However, at its meeting of 17 June 2010, the LLP Committee adopted the approach proposed 
by the Commission.  

The Commission stresses, however, that evidence from the external studies on the effects of 
Leonardo mobility for young trainees and graduates2 and the impact of VETPRO mobility on 
individuals and systems3 indicates that Leonardo mobility fulfils the programme objectives to 
a large extent. 

It  also  points  out  that  a  large  set  of  quantitative  data  are  available  in  the  Rap4Leo  and 
LLPLink databases. Rap4Leo in addition contains qualitative feedback from participants that 
shows a very high degree of satisfaction with their Leonardo experiences.  

2 
3 

http://ec.europa.eu/education/pdf/doc218_en.pdf 
Final report will be published soon.  

EN 

7 

   EN 

                                                 
49. In order to improve the quality and coherence of annual reporting by National Agencies, 
the Commission has provided them with extensive guidelines thereon in 2009. 

50.  With  regard  to  the  development  of  impact  indicators  for  the  Lifelong  Learning 
Programme, the LLP Committee adopted the revised approach proposed by the Commission 
at its meeting of 17 June 2010.  

51-52. Since the start of the LLP, the Commission continued to gather quantitative data and 
qualitative  feedback  from  participants  on  Leonardo  da  Vinci  mobility  through  Rap4Leo, 
resulting in an uninterrupted and reliable data series for the programme.  

In addition, the Commission underlines that, since the end of 2008, LLPLink has been used 
for  the  production  of  uniform  and  standardised  financial  reports  and  management  statistics. 
The  combination  of  the  statistical  data  from  Rap4Leo  and  LLPLink  ensure  the  quality  and 
coherence of data for Leonardo da Vinci mobility.  

Starting  in  2007,  the  Commission  has  been  extensively  discussing  with  the  programme 
management  committee  a  set  of  programme  impact  indicators  applicable  to  the  LLP  as  a 
whole.  The  committee  expressed  "no  opinion"  on  two  proposals  tabled  by  the  Commission 
and  asked  for  further  analysis  on  cost-effectiveness.  The  new  proposal  was  adopted  by  the 
committee  at  its  meeting  of  17  June  2010.  On  this  basis,  the  relevant  data  will  be  matched 
with the now established indicators; the first results of this exercise are expected by mid 2011. 

56.  The  Commission  is  aware  of  the  fact  that  many  National  Agencies  have  weaknesses  in 
their management of primary checks. This is pointed out formally to the National Authorities 
and National Agencies in the observations and recommendations issued after the analysis of 
the  yearly  National  Agency  reports,  as  well  as  following  monitoring  and  audit  visits  on  the 
spot. As primary checks are an important element for a reasonable assurance on the use of EU 
funds,  the  Commission  monitors  weaknesses  in  this  area  very  closely  and  sees  to  it  that 
appropriate remedial action is taken at national level. 

56. first indent.  

The Commission is aware that in one National Agency visited by the Court in the context of 
this audit the 45-day deadline for assessment and approval of the final reports is not respected. 
The Commission itself has made an observation and recommendation on this issue following 
its systems monitoring visit to the country in 2008. 

56. second indent.  

The example found by the Court of final payments to beneficiaries having been made without 
the required feedback from all mobility participants is confined to one National Agency only.  

58. The Commission has provided guidance to National Authorities on the required secondary 
controls since 2008. Based on the experience of the first two years of the programme and the 
Commission’s  own  supervisory  audits,  as  well  as  in  the  follow-up  of  the  Court’s 
recommendation  in  the  context  of  the  DAS  2008,  the  Commission  has  complemented  these 
guidelines in 2009 with specific procedures for secondary controls. 

CONCLUSION AND RECOMMENDATIONS 

EN 

8 

   EN 

59.  The  Commission  welcomes  the  Court’s  conclusion  that  it  established  an  appropriate 
project lifecycle management system for the Lifelong Learning Programme.  

The  Commission  took  into  consideration  not  only  the  findings  and  suggestions  from  the 
mandatory  evaluations,  but  also  those  from  external  studies.  However,  the  main  purpose  of 
external studies and reports is to provide the Commission with background information and 
analysis;  nevertheless,  their  status  is  substantially  different  from  the  mandatory  evaluations 
required  by  the  legal  basis.  Thus,  the  Commission  is  not  obliged  to  follow  them  up  in  the 
same formal way as mandatory evaluations. 

Although  the  Commission’s  IT  system,  LLPLink,  was  still  incomplete  at  the  time  of  the 
Court’s  audit,  as  its  development  started  in  2008,  the  system  is  currently  stable  and 
functioning properly, and the quality of data has been vastly improved by the introduction of 
electronic forms. At the same time, the parallel use of a specific tool for Leonardo mobility 
has ensured that no data loss has occurred, and the Commission has continued to collect all 
necessary data on the action without any gaps since the predecessor programme. 

60. The Commission welcomes the Court’s appreciation of operational elements such as the 
yearly  work  programme  of  the  National  Agencies,  information,  publicity  and  promotion  of 
the programme. 

The Commission discontinued its previous partner finding database due to quality assurance 
problems.  While  alternative  solutions  to  partner  search  mechanisms  are  currently  being 
analysed,  the  Commission  has  emphasized  that  the  lack  of  a  partner  search  database,  for 
example, does not prevent successful programme implementation as demand far exceeds the 
budget available to support Leonardo mobility projects. 

As  established  by  the  Commission’s  updated  guidelines  issued  in  2009,  a  quality  assurance 
check on the assessments of grant applications should be part of the secondary controls on the 
National Agencies by the National Authorities. 

Recommendation 1 

(a)  The  Commission  accepts  Recommendation  1  (a),  which  is  already  being  implemented: 
The  Commission  has  provided  National  Agencies  with  qualitative  feedback  since  the  2008 
annual  reporting.  Furthermore,  as  the  Court  recognised,  it  has  also  provided  feedback  on 
annual  work  programmes  for  those  cases  where  revision  was  needed  in  order  to  reach  the 
expected quality standards. 

(b) The Commission partly accepts Recommendation 1 (b). The Commission discontinued its 
previous  partner  finding  database  due  to  quality  assurance  problems.  While  alternative 
solutions  to  partner  search  mechanisms  are  currently  being  analysed,  the  Commission 
emphasizes that the lack of a partner search database, for instance, does not interfere with the 
successful  implementation  of  the  programme,  as  the  demand  by  far  exceeds  the  budget 
available to support Leonardo mobility projects. 

(c)  The  Commission  considers  that  National  Agencies  should  not  have  a  formal  role  of 
facilitating placements for participants from other countries as this would entail a major shift 
in  responsibilities  and  implications  for  the  legal  relationships  between  organisations  from 
other  countries.  This  will  not  prevent  them  from  having  an  exchange  of  suitable 
intermediaries on an informal basis. 

EN 

9 

   EN 

(d) The Commission accepts Recommendation 1(d). The Commission is working on further 
developing  the  assessment  handbook  for  evaluators  in  connection  with  the  new  on-line 
application and revised assessment form. 

(e) The Commission notes with regard to Recommendation 1(e) that it is currently revising its 
approach to the Quality and Impact monitoring visits, by integrating a small sample check of 
quality assessments of applications by National Agencies. Subsequently, the Commission will 
analyse the added value of such checks and their impact on resources from the point of view 
of the cost-effectiveness of controls  

(f) The Commission accepts Recommendation 1(f).  

61.  The  Commission  welcomes  the  Court’s  assessment  of  the  control  systems  in  place  at 
Commission and national levels. These control systems focus on providing assurance on the 
use of EU funds on a yearly basis. 

The  Commission  is  informed  of  the  programme  results  on  a  regular  basis  via  the  Yearly 
National Agency Reports on programme management and implementation as well as via its 
Quality and Impact Monitoring visits. 

The Commission stresses that it continued to gather quantitative data and qualitative feedback 
from  participants  on  Leonardo  da  Vinci  mobility  through  Rap4Leo,  resulting  in  an 
uninterrupted  and  reliable  data  series  on  these  aspects  since  the  start  of  the  programme. 
Pending a decision of the programme committee on the impact indicators it is not yet clear to 
what extent the data will fulfil the need of these indicators. 

The  Commission  underlines  that  impacts,  however,  can  only  be  measured  over  time.  The 
Member  States  are  expected  to  report  on  the  first  part  of  the  programme  in  their  interim 
evaluation due in 2010. This input will be timely for enhancing the performance of Leonardo, 
both in the current programme and for the preparation of the new programme generation. 

In addition, the Commission underlines that, since the end of 2008, LLPLink has been used 
for  the  production  of  uniform  and  standardised  financial  reports  and  management  statistics. 
The  combination  of  the  statistical  data  from  Rap4Leo  and  LLPLink  ensure  the  quality  and 
coherence of data for Leonardo da Vinci mobility.  

Starting  in  2007,  the  Commission  has  been  extensively  discussing  with  the  programme 
management  committee  a  set  of  programme  impact  indicators  applicable  to  the  LLP  as  a 
whole.  The  Committee  expressed  "no  opinion"  on  two  proposals  tabled  by  the  Commission 
and  asked  for  further  analysis  on  cost-effectiveness.  The  new  proposal  was  adopted  by  the 
committee  at  its  meeting  of  17  June  2010.  On  this  basis,  the  relevant  data  will  be  matched 
with the now established indicators; the first results of this exercise are expected by mid 2011.  

The Commission based on the above elements as well as on the impact studies considers that 
it has sufficient assurance on the effectiveness of the programme. 

Recommendation 2 

(a)  The  Commission  accepts  Recommendation  2  (a)  and  underlines  that  the  evaluation 
conclusions  of  the  annual  activity  reporting  of  the  National  Agencies  are  duly  integrated  in 
the Commission’s monitoring visits of National Agencies. 

EN 

10 

   EN 

(b) The Commission accepts Recommendation 2 (b) and will align the annual activity report 
structure  with  that  of  the  National  Agency  work  programmes  after  the  revision  of  the  work 
programme structure due in 2011. 

(c) The Commission accepts Recommendation 2(c). The Commission's revised proposal was 
adopted by the LLP Committee at its meeting of 17 June 2010. On this basis, the relevant data 
will  be  matched  with  the  now  established  indicators;  the  first  results  of  this  exercise  are 
expected by mid 2011.  

(d) The Commission accepts Recommendation 2 (d) and will endeavour to establish a system 
for measuring impact of the successor programme in a timely manner. 

The Commission underlines that impact cannot be measured from the outset, but only after a 
certain  period  of  programme  implementation.  This  is  normally  done  on  the  basis  of  studies 
and evaluations. The fact that the present legal basis of the LLP contains 58 objectives was a 
particular factor of complexity and did not facilitate the task of defining suitable indicators.  

62. The Commission welcomes the overall conclusion of the Court of Auditor's report that the 
design  and  management  of  the  mobility  scheme  of  the  Leonardo  da  Vinci  programme  is 
instrumental to achieving effective results.  

It has already started work and made progress on several points highlighted by the Court, such 
as  qualitative  feedback  regarding  the  annual  programming  of  work,  the  assessment  of 
applications and the improvement of the IT tool. 

The Commission accepts that the work on performance indicators needs to be finalised. 

It underlines, however, that the work with the programme committee on the establishment of 
indicators started already in 2007. Furthermore, based on the impact studies and evaluations 
of  the  previous  Leonardo  programme  and  the  available  quantitative  data  and  the  qualitative 
feedback  from  participants  in  the  current  programme,  the  Commission  is  confident  that  the 
Leonardo da Vinci mobility programme is producing effective results. 

EN 

11 

   EN