Special Report No 4/2010 ‘Is the design and management of the mobility scheme of the Leonardo da Vinci programme likely to lead to effective results?’
Date: 2010-09-29
Institutions: European Court of Auditors
Subjects: administration of the Institutions, management accounting, management audit, vocational training

european 
court of audItors

4
3
8
0
-
1
3
8
1
n
s
s

I

0
1
0
2

4
o
n

t
r
o
p
e
r

l

a
i
c
e
p
s

Is the desIgn and management of the  
mobIlIty scheme of the leonardo da vIncI 
programme lIkely to lead to effectIve 
results?

en

 
 
 
 
special r eport no  4 

 2010

Is the desIgn and 
management of the 
mobIlIty scheme of 
the leonardo da vIncI  
programme lIkely 
to lead to effectIve 
results?

(pursuant to article 287(4), second subparagraph, tfeu)

european court of audItors

european court of audItors  
12, rue alcide d e g asperi 
1615 luxembourg 
luXembourg

tel. +352 4398-1 
fax +352 4398-46410 
e-mail: euraud@eca.europa.eu 
Internet: http://www.eca.europa.eu

special r eport no 4 

 2010

a great deal of additional information on the european union is available on the Internet.

It can be accessed through the europa server (http://europa.eu).

cataloguing data can be found at the end of this publication

luxembourg: publications office of the european union, 2010

Isbn 978-92-9207-796-9

doi:10.2865/76038

© european union, 2010

reproduction is authorised provided the source is acknowledged.

Printed in Luxembourg

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

contents

3

Paragraph

I–V 

EXECUTIVE SUMMARY

1–10 

INTRODUCTION

8 
9 
10 

ROlE  Of ThE  COMMISSION
ROlE  Of ThE  NATIONAl A UThORITY
ROlE  Of ThE  NATIONAl AgENCY

11–15 

AUDIT SCOPE AND  APPROACh

16–58 

ObSERVATION S

16–29 
17–22 

23–29 

30–37 
31 

32 

33–35 

36–37 
38–52 
39–47 

48–52 

53–58 
54–56 

57 

58 

DESIgN
 the  commIssIon took account of  Its own mandatory evaluatIons of the predecessor 
programme, but dId not systematIcally take account of other  Important studIes and reports
 the  commIssIon establIshed an approprIate system for project lIfe cycle management.  the 
supportIng It  system, however, had some sIgnIfIcant omIssIons
MANAgEMENT
 arrangements for approvIng the natIonal agencIes’ annual work programmes are approprIate, 
although the commIssIon does not systematIcally provIde the  natIonal  agencIes wIth 
qualItatIve feedback thereon
 the  commIssIon and partIcIpatIng countrIes publIcIse and promote the programme 
satIsfactorIly
 the  commIssIon has not addressed applIcants’ dIffIcultIes 
countrIes
weaknesses  In the assessment of applIcatIons
REPORTINg
 there are shortcomIngs  In the system for reportIng on the results and effects of the 
leonardo programme
 the  commIssIon had started but not yet completed the establIshment of a comprehensIve 
system for the measurement of the Impact of  leonardo by the thIrd year of the programme
CONTROlS
 In most cases, control over leonardo by  natIonal  agencIes complIed wIth  commIssIon 
guIdance
 although the secondary controls carrIed out by  natIonal  authorItIes generally provIde 
reasonable  assurance that the prImary controls are effectIve, a number of weaknesses were 
IdentIfIed
 the  commIssIon only recently provIded complementary guIdance on the specIfIc procedures 
covered by the secondary controls

In fIndIng host partners  In other 

59–62 

CONCl USIONS AND RECOMMENDATIONS

ANNEX — ObjECTIVES (bASED ON ThE lEgAl bASE Of ThE lIfElONg lEARNINg PROgRAMME) 

REPlY Of ThE COMMISSION

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
4

eXecutIve  
summary

I.
t h e   l e o n a r d o   d a   v i n c i   p r o g r a m m e   i s 
d e s i g n e d   t o   i m p l e m e n t   t h e   e u r o p e a n 
union’s vocational education and training 
polic y. m obilit y  proje c ts, th e larg est par t 
o f   t h e   p ro g r a m m e,   e n a b l e   o rg a n i s a t i o n s 
i n v o l v e d   i n   v o c a t i o n a l   e d u c a t i o n   a n d 
t r a i n i n g   t o   s e n d   p a r t i c i p a n t s   t o   a n o t h e r 
e u ro p e a n   c o u n t r y,   t h u s   g i v i n g   t h e m   t h e 
c h a n c e   t o   i m p r o v e   t h e i r   c o m p e t e n c e s , 
k nowle dge and  sk ills. 

II.
t h e   e u r o p e a n   c o m m i s s i o n   h a s   o v e r a l l 
r e s p o n s i b i l i t y   f o r   t h e   p r o g r a m m e   a n d 
m o n i to r s   a n d   s u p e r v i s e s   i t s   i m p l e m e nt a-
tion in cooperation with national authori-
t i e s   —   u s u a l l y   e d u c at i o n   m i n i s t r i e s   —   i n 
e a c h   p a r t i c i p a t i n g   c o u n t r y.   op e r a t i o n a l 
r e s p o n s i b i l i t y   f o r   m a n a g i n g   m o b i l i t y 
p ro j e c t s   i s   f u l l y   d e vo l ve d   to   t h e   n at i o n a l 
a g e n c i e s   d e s i g n a t e d   b y   t h e   m e m b e r 
states.

III.
the cour t considered whether the design 
a n d   m a n a g e m e n t   o f   t h e   li fe l o n g   le a r n-
i n g   pro g r a m m e’s   ( 2 0 0 7 – 1 3 )   le o n a rd o   d a 
vi n c i   m o b i l i t y   s c h e m e   i s   l i k e l y   to   l e a d   to 
effec t ive results.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

5

eXecutIve  
summary

—  controls at national agenc y level com-
plied with  c ommiss ion  guidan ce. 

—  t h e r e   a r e   w e a k n e s s e s   i n   t h e   w a y 
 n a t i o n a l   a u t h o r i t i e s   c a r r y   o u t   t h e i r 
s e c o n d a r y   c o n t ro l s   o f   n a t i o n a l   a g e n-
c ies. 

—  th e   ove ra l l   co n c l u s i o n   i s   t h at   t h e   d e -
sign an d manag ement  of th e mo bili t y 
sc he me  of  th e  leon ardo  da vin ci  pro -
gramme could lead to effective results. 
t h e   c o m m i s s i o n   h a d   s t a r t e d   b u t   n o t 
yet   comple ted  t he  est ablis hment  o f  a 
c o m p r e h e n s i v e   s y s t e m   fo r   t h e   m e a s-
urement of the impac t of leonardo. as 
a   r e s u l t ,   t h e   c o m m i s s i o n   c a n n o t   y e t 
a s s e s s   h o w   t h e   o b j e c t i v e s   a r e   b e i n g 
met three years into the programme — 
around  half of its lifespan.

V.
on   t h e   b a s i s   o f   t h e s e   o b s e r v a t i o n s ,   t h e 
c o u r t   m a k e s   r e c o m m e n d a t i o n s   t o   t h e 
c o m m i s s i o n   t o   i m p r o v e   t h e   s u p p o r t i n g 
It  s y s t e m ,   p r o v i d e   q u a l i t a t i v e   f e e d b a c k 
r e g a r d i n g   t h e   a n n u a l   p r o g r a m m i n g   o f 
w o r k ,   p r o v i d e   m o r e   s u p p o r t   f o r   f i n d-
i n g   h o s t   p a r t n e r s ,   f u r t h e r   d e v e l o p   a n d 
m o n i to r   p ro ce d u re s   fo r   a s s e s s i n g   p ro j e c t 
a p p l i c a t i o n s ,   f i n a l i s e   s m a r t  o b j e c t i v e s 
a n d   p e r fo r m a n ce   i n d i c ato r s   w i t h o u t   f u r -
t h e r   d e l a y,   i m p r o v e   t h e   a s s e s s m e n t   o f 
r e s u l t s ,   f o c u s i n g   m o r e   o n   t h e   i m p a c t   o f 
the programme, and, in the future, ensure 
t h a t   fo r   a ny   s u cce s s o r   p ro gr a m m e   a   s ys-
te m   fo r   m e a s u r i n g   i m p a c t   i s   s e t   u p   f ro m 
th e outse t. 

IV.
t h e   c o u r t ’s   f i n d i n g s   r e l a t e   t o   a n   a u d i t 
c a r r i e d   o u t   at   t h e   co m m i s s i o n   a n d   i n   s i x 
p a r t i c i p a t i n g   c o u n t r i e s   s e l e c t e d   f o r   t h e 
a u d i t.   si n ce   le o n a rd o   i s   o n e   o f   fo u r   e d u-
cation and training programmes included 
i n   t h e   c o m m i s s i o n’s   li f e l o n g   l e a r n i n g 
programme, many audit findings have rel-
e va n ce   t h at   ex te n d   to   t h e   o t h e r   l i fe l o n g 
le a r n i n g   pro gra m m e   s c h e m e s.  th e   co u r t 
found t hat :

— 

I n   d e s i gn i n g   t h e   c u r re nt   le o n a rd o   d a 
vinci programme (2007–13), the com-
mis sio n  to ok  accou nt   o f  it s  own   man-
dator y evaluations of the pre decessor 
programme, but did not systematically 
take account of other impor tant stud-
ies  and  rep or ts.

—  the commission established an appro -
p r i a t e   p ro j e c t   l i fe   c yc l e   m a n a g e m e n t 
system for leonardo, but the suppor t-
i n g   I t   s y s t e m   h a d   s o m e   s i g n i f i c a n t 
om issio ns.

—  th e   o p e r a t i o n a l   e l e m e n t s   o f   t h e   p ro -
gramm e,   inc ludin g  p ub l ic it y  and   pro -
m o t i o n   o f   t h e   p r o g r a m m e   a n d   t h e 
annu al  p ro grammin g c ycl e,  we re ge n-
erally well managed, but the commis-
s i o n   d i d   n o t   a d d r e s s   t h e   d i f f i c u l t i e s 
o f   a p p l i c a n t s   i n   f i n d i n g   p a r t n e r s   a n d 
did  no t ensure  t he q ual it y of n at ion al 
a g e n c i e s’  a s s e s s m e nt s   o f   gra nt   a p p l i -
cat io n s.

—  a system of repor ting is in place but it 
does not yet enable the commission to 
measure the results and effec ts of the 
p r o g r a m m e .   t h e r e   a r e   s h o r t c o m i n g s 
in  th e  s ys te m  fo r  repo r t in g  o n  th e   re -
sul ts  and effec ts  o f the leo nardo pro -
gramm e.  the   co mm issio n  ha d  star te d 
b u t   n o t   ye t   c o m p l e t e d   t h e   e s t a b l i s h-
m e n t   o f   a   c o m p r e h e n s i v e   s y s t e m   f o r 
t h e   m e a s u r e m e n t   o f   t h e   i m p a c t   o f 
leo nardo by the third year of the pro -
gramm e.   

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

IntroductIon

6

1. 

2. 

3.  

4. 

 th e   li s b o n   eu ro p e a n   co u n c i l   i n   m a rc h   2 0 0 0   s e t   t h e   s t r a te gi c 
obj ec t ive for  th e eu rop ea n union to become t he wor ld ’s mos t 
dynamic k nowledge -based economy by 2010. I t recognised the 
i m p o r t a n t   r o l e   o f   e d u c a t i o n   a s   a n   i n t e g r a l   p a r t   o f   e c o n o m i c 
and social policies. the basis for european cooperation in voca-
tional education and training was laid down in the subsequent 
‘cope nhag en  p ro ces s’  1  whic h   es tablished  th e  obj ec tive   of   in-
c r e a s i n g   v o l u n t a r y   c o o p e r a t i o n ,   i n   o r d e r   t o   p r o m o t e   m u t u a l 
trust, transparenc y and recognition of competences and quali-
f i c at i o n s,   t h e re by   e s t a b l i s h i n g   a   b a s i s   fo r   i n c re a s i n g   m o b i li t y 
an d fa cil itat ing  acces s  to  l ifel on g  le ar n in g.

 the  euro p ean   co mmis sio n  ha s  brought  its  var ious  e duc at ion-
a l   a n d   t r a i n i n g   i n i t i a t i v e s   t o g e t h e r   u n d e r   a   s i n g l e   u m b r e l l a , 
t h e   li fe l o n g   le a r n i n g   pro g r a m m e 2,   w i t h   t h e   a i m   o f   fo s t e r i n g 
 interchange, cooperation and mobilit y bet ween education and 
training   systems   w ith in  th e  union .  th e  new  lifelon g  lear nin g 
pro g r a m m e   re p l a c e s   p re v i o u s   e d u c a t i o n ,   vo c a t i o n a l   t r a i n i n g 
and  e lear ning  p rogrammes, wh ich  e nde d  in  2 0 0 6 .

 an   i n d i c at i ve   f i n a n c i a l   a l l o c at i o n 3  fo r   t h e   i m p l e m e nt at i o n   fo r 
the seven-year period star ting on 1 januar y 2007 was 6  970  mil-
l i o n   e u ro.  wi t h i n   t h i s   a l l o c a t i o n ,   t h e   s u m s   t o   b e   a l l o c a t e d   t o 
the  su bp ro gram mes  shal l  n ot be  le ss  t ha n:

1  the copenhagen process was 
launched in november 2002 by 

the ministers for education and 

vocational training of the european 

union and the countries belonging 

to the european economic area 

(eea) and the european free 

trade area (efta), the european 

social partners and the european 

commission.

2  decision no 1720/2006/ec of 
the european parliament and of 

the council of 15 november 2006 

establishing an action programme 

in the field of lifelong learning 

(oj l 327, 24.11.2006, p. 45).

3  article 14 and paragraph 
b.11 of the annex to decision 

no 1720/2006/ec. 

4 

In this report, the term ‘leonardo’ 

refers to the current phase of the 

programme.

—  c o men ius  13 %;

—  eras mus 4 0 %;

—  l eo nard o da vinci 2 5 %;

—  grund t v ig  4 %.

 l e o n a r d o   d a   v i n c i   i s   t h e   n a m e   g i v e n   t o   t h e   s u b p r o g r a m m e 
designed to implement the european union’s vocational educa-
tion and training polic y, suppor ting and supplementing actions 
t a k e n   b y   t h e   m e m b e r   s t a t e s .  t h e   c u r r e n t   p h a s e   o f   t h e   le o n-
ardo   da vinci  p ro gramme   2 00 7–1 3   is  a  co nt in uat ion   o f  for me r 
programmes, leonardo I (1995–99) and leonardo II (2000–06). 
the general, specific and operational objec tives of the lifelong 
le a r n i n g   pro gra m m e   a n d   o f   le o n a rd o   d a  vi n c i     a re   s e t   o u t   i n 
paragrap h 44  a nd in t he A n nex.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
  
 
7

5.  

 a  r a n g e   o f   a c t i v i t i e s   m a y   b e   s u p p o r t e d   b y   t h e   l e o n a r d o   d a 
vin ci p ro gramme, incl u ding :

—  mobility — where par ticipants spend some time in another 

euro p ean  cou nt r y ; 

—  p ar tn erships  fo cus ing o n t hem e s of  mutu al i nte re st ; 

—  multilateral projec ts, in par ticular those aimed at improv-

ing  t raining  systems.

6. 

 m ob il it y  p ro jec t s  cont in ue  to   be  on e  of   t he   mai n   for ms   of  ac -
tivit y in leonardo, as they were for its predecessors. as shown 
in  G ra p h  1,  t he   sh are  of   leo nardo  grants   awarded  to  mobili t y 
pro jec t s  wa s 64  % in 2007 and 5 9  %  in  2 0 0 8 . 

g R A Ph   1

l E O N A R D O   D A  VI N C I ,  gR A N T S   AwA R D E D  TO   P R OjE C T S   bY  T Y P E   Of 
AC T I V I T Y  2007 – 0 8  (M Il lI O N   E U R O)

2007

2008

0

50

100

150

200

250

Mobility

Partnerships

Multilateral

Source: european commission.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

  
 
8

7. 

 m ob il it y p ro jec ts s up p o r ted by  leon ardo are inten ded  to give 
individuals, including trainees in initial vocational training and 
people already in the labour market, as well as vocational edu-
cation and training professionals, the chance to improve their 
co m p e te n ce s,   k n ow l e d g e   a n d   s k i l l s   t h ro u g h   a   t ra i n i n g   p l a ce -
ment in another european countr y. grants contribute to project 
management, travel costs and living expenses. the distribution 
of p ar tic ip ant s  in 2008 mo b il i t y  proj e c ts  by fi el d of  e du c ati on 
and  training  is  s hown  in G ra ph  2.

g R A Ph   2

PA R T I C I PA N T S   I N  2008  M ObIlI T Y   P R OjE C T S   bY   fI ElD   Of 
E D U C AT I O N   A N D  T R A I N I Ng 

Other fields
27 %

Education
19 %

Humanities and arts
4 %

Social sciences,
business and law
11 % 

Science,mathematics
and computing 
3 %

Engineering,
manufacturing and
construction
 11 %

Services
16 %

Health and welfare
6 %

Agriculture and
veterinary
3 % 

Source: european commission. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
9

R OlE   Of   ThE   CO M M I S S I O N

8. 

 t h e   e u r o p e a n   c o m m i s s i o n   h a s   o v e r a l l   r e s p o n s i b i l i t y   f o r   e n -
s u r i n g   e f fe c t i ve   a n d   e f f i c i e nt   i m p l e m e nt at i o n   o f   t h e   li fe l o n g 
learning programme 5. the lifelong learning programme com-
m i t te e,   co m p r i s i n g   re p re s e n t a t i ve s   o f   m e m b e r   s t a te s,   a s s i s t s 
t h e   co m m i s s i o n   w i t h   t h e   i m p l e m e n t a t i o n   o f   t h e   p ro gr a m m e. 
the  c o mm issio n’s ro le is to:

5  article 6 of decision 
no 1720/2006/ec.

6  article 15 of decision 
no 1720/2006/ec.

— 

regular ly monitor and evaluate the lifelong  lear ning pro -
g r a m m e   a g a i n s t   i t s   o b j e c t i v e s 6  i n   c o o p e r a t i o n   w i t h   t h e 
m em b er s t ate s;  and

—  es tab l is h ap prop r iate s upe r viso r y  controls,  by  ove r se e in g 
a n d   c o o rd i n a t i n g   t h e   o p e r a t i o n   o f   t h e   w h o l e   s ys t e m   a n d 
rev ie w in g t he systems  of  nationa l co nt rols 7. 

7  article 8 of commission decision 
of 26 april 2007 relating to the 

respective responsibilities of the 

member states, the commission 

and the national agencies in 

implementation of the lifelong 

learning programme (2007–13).

R OlE   Of   ThE   NAT I O N Al   A U ThO R I T Y

9. 

 t h e r e   a r e   3 1   p a r t i c i p a t i n g   c o u n t r i e s   i n   t h e   p r o g r a m m e :   t h e 
27 e u m em b er s t ate s p l us I celand,  liec htenstein , nor way  and 
tur k ey. the  n at ional  aut hor it y, usually th e ed uc ati on  mi n ist r y 
in each par ticipating countr y, monitors and super vises the work 
of its national agenc y in cooperation with the commission. the 
n at i o n a l   a u t h o r i t y   i s   re s p o n s i b l e   fo r   e s t a b l i s h i n g   a   s ys te m   o f 
secon dar y   co nt rol s,   w hos e  o bj ec ti ve  is   to  give   rea son able   as-
s u ra n ce   t h at   t h e   s ys te m s   a n d   p r i m a r y   co nt ro l s   c a r r i e d   o u t   by 
th e natio nal ag e nc ies are wo r k i n g  ef fec t ivel y. 

R OlE   Of   ThE   NAT I O N Al   AgE N C Y

10.  

 operational responsibility for mobility projects within  leo nardo 
is fu l ly de vo lved to  natio nal  a ge nc ie s  e st abli s he d wit h in  e ach 
par ticipating countr y. these national agencies are responsible 
for  t he  ma nagement  of  th e  l ife  c ycle   of   th e   mobil it y   proj ec t s. 
they also contribute, as required, to the commission’s monitor-
ing and evaluation of the programme. the func tions expec ted 
o f   t h e m   a r e   s e t   o u t   i n   t h e   l e g a l   b a s i s   o f   t h e   p r o g r a m m e ,   i n -
c l u d i n g   t h e   co m m i s s i o n   d e c i s i o n   o n   t h e   re s p e c t i ve   re s p o n s i-
b i l i t i e s   o f   t h e   eu ro p e a n   co m m i s s i o n ,   t h e   m e m b e r   s t a t e s   a n d 
t h e   n a t i o n a l   a g e n c i e s   i m p l e m e n t i n g   t h e   li f e l o n g   l e a r n i n g 
pr o g r a m m e   ( c( 2 0 0 7 )   1 8 0 7 ) ;   t h e   o p e r a t i o n a l   i n s t r u c t i o n s   a r e 
detail e d  in  t he ‘guid e  fo r  nat ional  agenc ies’  ,  wh ic h  i s  par t   of 
the yearly financial agreement signed between the commission 
and  th e natio na l agen cies.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
audIt scope and approach

10

11.  

 the audit focused on mobility activities funded by the leonardo 
p ro g r a m m e.  t h e   o ve r a l l   o b j e c t i ve   w a s   t o   a s s e s s   w h e t h e r   t h e 
design and management of the lifelong learning programme’s 
(20 07–1 3)   leo nardo  da vinci  m obilit y  sch eme  is  likely  to  l ead 
to ef fec tive res ul ts. 

8  france, germany, malta, poland, 
romania and the united kingdom.

12.  

the  o b jec tives o f the au dit were  to  ass e ss  wh et h er :

(a)  t h e   d e s i g n   o f   t h e   p r o g r a m m e   t o o k   a c c o u n t   o f   r e l e v a n t 
e v a l u a t i o n s   a n d   s t u d i e s   a n d   h a d   a n   a p p r o p r i a t e   p r o j e c t 
l ife  c ycl e manag ement s ys tem; 

(b)   the operational elements of the programme, including the 
p r e p a r a t i o n   o f   a n n u a l   w o r k   p r o g r a m m e s ,   t h e   p r o c e s s   o f 
publicit y, promotion and provision of infor mation and se -
l ec t io n o f pro jec ts  to be fun ded,  we re  well  m an ag e d;

(c)  t h e re   w a s   a   re p o r t i n g   s ys te m   i n   p l a ce   w h i c h   e n a b l e d   t h e 
commission to measure the results and impac t of the pro -
gramm e; and

(d)  t he s ys tem o f co nt rol was  adequ ate.

13.  

14.  

15.  

 t h e   a u d i t   c o v e r e d   t h e   p r e p a r a t i o n   o f   t h e   li f e l o n g   l e a r n i n g 
pro g r a m m e,   w h i c h   s t a r t e d   i n   2 0 0 4 ,   a n d   t h e   a c t u a l   e xe c u t i o n 
of the programme in 2007–08. r elevant developments in 2009 
h ave   a l s o   b e e n   t a k e n   i n to   a cco u nt .   I n   a d d i t i o n   to   a n   a u d i t   a t 
the commission, six member states 8 were visited. 45 % of total 
le o n a rd o   m o b i l i t y   f u n d s   we re   a l l o c a te d   to   t h e m   i n   2 0 0 7   a n d 
44 %  in 2008 (see Gra p h  3). 

 since  leonardo is par t of an integrated lifelong  lear ning pro -
gramme with various educational and training initiatives under 
a sing l e umb rell a, many  f indin gs,  con clusi ons  and  re comm en-
datio ns  ap p l y to t he p ro gramme   as  a wh ole. 

 the audit work included the collec tion and analysis of data, re -
views of planning, evaluation and implementation documents, 
an examination of mobility projec t files and inter views with of-
ficials from the commission, national author ities and national 
agencies and with representatives of beneficiar y organisations 
in t he  sel ec te d m em be r s t ate s.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
 
11

g R A Ph   3

VAl U E   Of   l E O N A R D O   M ObIlI T Y   gR A N T S   bY   CO U N T R Y   AwA R D E D 
2007 – 08  (M Il lI O N   E U R O)

0

5

10

15

20

25

DE
FR
IT
ES
UK
PL
TR
NL
EL
PT
CZ
RO
HU
BE
AT
BG
SE
SK
FI
DK
NO
IE
LT
LV
SI
EE
CY
MT
IS
LU
LI

2007

2008

Source: european commission.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

9  basic requirements on the 
scope, purpose, timing and use of 

evaluations are set in the financial 

regulation (articles 27(4), 28, 33, 

56(3) and 166) and its Implementing 

rules (article 21).

observatIons

12

DESIgN

16.  

 t h e   a u d i t   e x a m i n e d   w h e t h e r   t h e   c o m m i s s i o n   t o o k   a c c o u n t , 
when it established the leonardo programme, of the wealth of 
infor mation available to it about its predecessor programmes, 
i . e .   a l l   e v a l u a t i o n s ,   s t u d i e s   a n d   n a t i o n a l   r e p o r t s   i n c l u d i n g 
whether national authorities made use of the national repor ts. 
I t   co n s i d e re d   w h e t h e r   t h e   co m m i s s i o n   e s t a b l i s h e d   a n   a p p ro -
p r i a t e   p r o j e c t   l i f e   c y c l e   m a n a g e m e n t   s y s t e m   a t   c o m m i s s i o n 
a n d   p a r t i c i p a t i n g   c o u n t r y   l e v e l   f o r   i m p l e m e n t i n g   l e o n a r d o, 
i n c l u d i n g   t h e   s u p p o r t i n g   It  s y s t e m .   a t   c o m m i s s i o n   l e v e l ,   i t 
assessed whether there was a clear assignment of tasks to dif-
f e r e n t   u n i t s .   a t   p a r t i c i p a t i n g   c o u n t r y   l e v e l ,   i t   e x a m i n e d   t h e 
administrative implementation by the national authorities and 
a g e n c i e s   c h a rg e d   w i t h   t h e   p ro gra m m e’s   m a n a g e m e nt,   w h o s e 
ro l e s   a n d   re s p o n s i b i l i t i e s   a re   s e t   o u t   i n   g u i d e l i n e s   i s s u e d   b y 
the  c o mm issio n.

T hE   CO M M I S S I O N  TO Ok   ACCO U N T   Of   I T S   O wN  
M A N D ATO R Y   E VAl UAT I O N S   Of   ThE   P R E D E C E S S O R 
P R OgR A M M E ,  bU T   D I D   N OT   S YS T E M AT I C Al l Y  TAkE  
ACCO U N T   Of   OThE R   I M P O R TA N T   S T U D I E S   A N D 
R E P O R T S

17.  

 a l l   p r o g r a m m e s   f u n d e d   f r o m   t h e   g e n e r a l   b u d g e t   o f   t h e 
 europ ea n  unio n  are  subj ec t  to  man dator y  evaluati on  at  t hre e 
stag es 9:

—  E x   a n t e  e v a l u a t i o n :   a   r e p o r t   a c c o m p a n i e s   t h e   l e g i s l a t i v e 

p ro po s al fo r a n ew  or rene wed  e u pro gra mm e  or  ac ti on .

—  m i d - t e r m   e v a l u a t i o n :   c a r r i e d   o u t   a t   t h e   h a l f w a y   s t a g e   o f 
a   p r o g r a m m e ,   t h i s   p r o v i d e s   d i r e c t   f e e d b a c k   d u r i n g   t h e 
co u r s e   o f   a   p ro gra m m e   a n d   c a n   t h u s   h e l p   to   i m p rove   t h e 
q u a l i t y   o f   o n g o i n g   i n t e r v e n t i o n s .   m o r e o v e r,   g i v e n   t h e 
lengthy lead time involved in setting up new programmes, 
m i d - t e r m   e v a l u a t i o n s   a re   a l s o   ve r y   i m p o r t a n t   s o u rc e s   o f 
information for the design of the nex t generation of a pro -
gramm e.

—  f inal  eval uatio n:  at the end of a progra mme.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
13

10  joint report on the 
final evaluation of socrates II, 

leonardo da vinci II and 

elearning, 2008.

18.  

19.  

20.  

21.  

 I n   d e s i g n i n g   t h e   n e w   le o n a rd o   p ro g r a m m e,   t h e   co m m i s s i o n 
t o o k   a c c o u n t   o f   t h e   r e s u l t s   o f   t h e   m i d - t e r m   a n d   f i n a l   e v a l u-
atio ns  o f  t he  predeces so r  p rogra mm e.  th e  results   of   th e  mid-
ter m evaluation were taken into account in the preparation of 
th e ex a nte evalu at ion .  fo r b o th t he mid- ter m and f in al evalu-
ations, the commission prepared internal ac tion plans, setting 
o u t   h o w   t h e   r e c o m m e n d a t i o n s   o f   e a c h   o n e   w o u l d   b e   t a k e n 
into account during the initial design phase and in subsequent 
adjus t ments of  t he design. howe ver, th e inte r nal ac tion plans 
for the establishment of It and  smart objec tives and per form-
ance indicators have not yet brought the expec ted results (see 
paragrap hs 27–29 and 48–52).

 ea c h   p a r t i c i p a t i n g   c o u n t r y   i s   r e q u i r e d   t o   p r o d u c e   i t s   o w n 
n a t i o n a l   m i d - t e r m   a n d   f i n a l   r e p o r t s   o n   t h e   i m p l e m e n t a t i o n 
a n d   e f f e c t i v e n e s s   o f   t h e   p r o g r a m m e ,   a n d   i t s   i m p a c t   o n   t h e 
voc at io nal  training   s ys tems  and   a r ran g em ents   in   th e   m ember 
st ate s,   a n d   to   s u b m i t   t h e m   to   t h e   co m m i s s i o n .  th e s e   re p o r t s 
must  b e su bm it ted  to the co mmiss ion  in  t he  contex t of mo ni-
t o r i n g   a n d   e v a l u a t i o n   o f   t h e   p r o g r a m m e ,   a n d   a n a l y s e d   a n d 
foll owed  u p in t hat  co ntex t. 

 analysis of the final national repor ts was entrusted to an ex ter-
nal contrac tor hired by the commission to under take the final 
eval uat io n 10.   howeve r,  t he  co ntrac tor  was  n ot   able   to   an a lys e 
all the national evaluation repor ts, as only 19 of the 31 par tici -
pating countries sent them to the commission by the required 
d e a d l i n e.   fo r   i t s   p a r t,   t h e   co m m i s s i o n   d i d   n o t   u n d e r t a k e   a ny 
s e p a r a t e   a n a l y s i s   o f   t h e   n a t i o n a l   r e p o r t s   p r o v i d e d   a f t e r   t h e 
dea dl ine, o r o t her w is e fol l ow  th e m up.

 th e   u s e   m a d e   o f   t h e s e   n at i o n a l   re p o r t s   by   s o m e   n at i o n a l   a u-
t h o r i t i e s   w a s   a l s o   l i m i te d.   I n   t h re e   o f   t h e   m e m b e r   s t a te s   v i s-
i t e d ,   t h e   r e c o m m e n d a t i o n s   w e r e   r e v i e w e d   a n d   f o l l o w e d   u p 
b u t   o n l y   o n e   o f   t h e m   i n c l u d e d   t h e   re co m m e n d at i o n s   i n   t h e i r 
management ar rangements for the new leonardo programme. 
I n  t he  o the r  three  co untr ies  visi te d,  th ere  was   no   e vi de n ce  of 
exa minat io n o f the rep o r ts  by  n ati on al au th or i ti es.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
14

22.  

 the  co mmis sio n has car r ied out t wo m aj or  st ud i es  on  as pe c ts 
of mobility ac tivities within leonardo 11. the commission states 
th at   t hese  stu dies  s up p or t   d ecis ion -m ake r s   in  th ei r   wor k ,   but 
i t   i s   n o t   c l e a r   h ow   t h e   co m m i s s i o n   i t s e l f   ex p l o i te d   t h e   u s e f u l 
i n fo r m a t i o n   t h e y   c o n t a i n e d   i n   o rd e r   t o   d e ve l o p   a n d   i m p rove 
the  effec tivenes s  of the p ro gram me.

T hE   CO M M I S S I O N   E S TAb lI ShE D   A N   A P P R O P R I AT E 
S YS T E M   fO R   P R OjE C T   lIfE   C YClE   M A N AgE M E N T . 
T hE   S U P P O R T I Ng   I T   S YS T E M ,  hO wE V E R ,  hA D   S O M E 
S IgN IfI C A N T   O M I S S I O N S 

23.  

 the audit encompassed the projec t life c ycle management sys -
t e m   s e t   u p   b y   t h e   c o m m i s s i o n   a n d   i t   a s s e s s e d   w h e t h e r   t h i s 
syste m is a ppro p r iate.  the  co ur t con sider s th is to be th e cas e 
i f   t h e   m a n a g e m e n t   o f   t h e   p ro j e c t   l i fe   c yc l e   i s   c l e a r l y   d i v i d e d 
b e t we e n   t h e   p a r t i e s   i nvo l ve d,   t h e   t a s k s   a re   c l e a r l y   d e s c r i b e d 
and  th ere are  s up er vis or y m easu re s i n  place.

11  ‘study on the obstacles to 
transnational mobility facing 

apprentices and other young people 

in initial vocational training and 

on ways of overcoming them’, also 

referred to as the move-it study 

(2007); ‘Impact analysis of leonardo 

da vinci mobility measures on young 

trainees and employees, and the 

influence of socio-economic factors’  

(2007).

b O X   1

EX A M PlE S   Of   R E CO M M E N D AT I O N S   fO R   AC T I O N   M A D E   I N  ThE  
‘ IM PAC T   AN Al YS I S   Of   l E O N A R D O   D A  VI N C I   MObIlI T Y   ME A S U R E S  
O N  YO U Ng  TR A I N E E S   A N D   EM Pl OY E E S ,  A N D  ThE   INf l U E N C E   Of 
SO C I O-E CO N O M I C   fAC TO R S’  ( 2 00 7 )

•	 there are still some weaknesses in the implementation of the measures in the host countries, and 
also in their preparation. better preparation of projects, including involving the participants, a more 
careful selection of host organisations and more and better mentoring/tutoring during the mobility 
measures by the sending and, in particular, the host organisation, are required. 

•	

Impact analysis of the programme should be carried out to a greater extent and on an ongoing basis. 
constant evaluation of the processes should look at preparation, implementation and follow-up, and 
especially the process of skill development (impact), while also observing participants’ needs.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
15

24.  

25.  

 following the launch of the new lifelong  learning programme 
i n   2 0 0 7 ,   t h e   co m m i s s i o n’s   di re c t o r a t e - g e n e r a l   fo r   e d u c a t i o n 
a n d   cu l t u re   re o rg a n i s e d   i t s e l f   to   re f le c t   t h e   n e e d s   o f   a   n e w ly 
inte grate d pro gramme. 

 previously, there were different national agencies dealing with 
different educational programmes. following the launch of the 
n e w   l i f e l o n g   l e a r n i n g   pr o g r a m m e   i n   2 0 0 7 ,   t h e   c o m m i s s i o n 
enco u rag ed  t he  p ar ticip at ing  st ate s  to  d es ign ate  a  sin gle   n a-
t i o n a l   a g e n c y   f o r   a l l   s u b p r o g r a m m e s   o f   t h e   li f e l o n g   l e a r n-
i n g   p r o g r a m m e ,   a n d   t h i s   i s   n o w   t h e   c a s e   i n   m o s t   c o u n t r i e s . 
m o b i l i t y   p r o j e c t s   u n d e r   l e o n a r d o   a r e   d i r e c t l y   a d m i n i s t e r e d 
by   n a t i o n a l   a g e n c i e s   w h i c h   a re   re s p o n s i b l e   fo r   a l l   a s p e c t s   o f 
manag eme nt :

- 

- 

- 

p rom o ting  t he p rogramm e;

o r g a n i s i n g   t h e   g r a n t   a w a rd   p ro c e d u re,   i n c l u d i n g   a s s e s s-
me nt  o f ap pl ic at ion s;

issuing grant agreements and transferring grant payments 
to  su ccess ful  ap p l icants;

-  mo nitor ing  and s up p o r ting  programme  ben efi ci ar ie s;

- 

- 

o rg a nising t he diss em ination  of  res ult s;

providing feedback on how the programme is func tioning 
and  having an imp ac t  in the ir count r y.

26.  

 national agencies are overseen by appointed national author -
i t i e s   w h i c h   a re   re s p o n s i b l e   fo r   p ro p e r   m a n a g e m e nt   o f   t h e   e u 
f u n d s   t r a n s fe r re d   t o   t h e   n a t i o n a l   a g e n c y   a n d   fo r   m o n i t o r i n g 
and super vising the work of the national agenc y in cooperation 
wi th  th e c o mm issio n. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
16

12  2008 guide for national agencies 
implementing the lifelong learning 

programme (paragraphs 28.1. and 

9.1.1.).

27.  

28.  

 I n general, therefore, the struc ture for the management of the 
p ro j e c t   l i fe   c yc l e   i s   a p p ro p r i ate.   howe ve r,   t h e re   we re   s i gn i f i-
cant weak nesses in the It system for projec t management and 
repor ting. national agencies are required to use the It systems 
made available by the commission 12. the most recent It system 
developed by the commission is llplink, a tool for the  lifelong 
le a r n i n g   pr o g r a m m e   d e s i g n e d   t o   m a n a g e   p r o j e c t   l i fe   c y c l e s 
e l e c t r o n i c a l l y,   f r o m   a p p l i c a t i o n   t o   f i n a l   p a y m e n t .   I n   p a r a l l e l 
with llplink , the  commission was also developing web -based 
e - f o r m s   t o   e n a b l e   a p p l i c a t i o n s   t o   b e   s u b m i t t e d   o n l i n e ,   a n d 
t o   e n a b l e   n a t i o n a l   a g e n c i e s   t o   a s s e s s ,   s e l e c t   a n d   r e p o r t   o n 
projects online. national agencies were required to use llplink 
for al l  n ew  p roj ec ts fro m 2008 onwards. 

 the new suppor ting It system had some significant omissions. 
at   t h e   t i m e   o f   t h e   a u d i t ,   l l p li n k   c o u l d   s a t i s f y   o n l y   t h e   m o s t 
u r g e n t   l i f e l o n g   l e a r n i n g   p r o g r a m m e   n e e d s   b u t   t h e   p r o j e c t 
was late and incomplete. the national agencies that wished to 
man ag e  t he  app lic atio n  as sessm e nt   proce ss   and  repor t in g   by 
using online solutions had to continue to use the old system or 
their local systems in parallel with llplink throughout 2009. In 
cases where there were no inter faces bet ween llplink and the 
s y s t e m s   u s e d   b y   t h e   n a t i o n a l   a g e n c i e s,   d a t a   h a d   t o   b e   i n p u t 
t w i c e ,   w h i c h   w a s   t i m e - c o n s u m i n g   a n d   p l a c e d   a n   a d d i t i o n a l 
burd en o n s t aff. 

29.  

 delays in the implementation of llplink have an impac t on the 
m a n a g e m e n t   o f   le o n a rd o   a n d   t h e   re s t   o f   t h e   li fe l o n g   le a r n-
ing programme since the system cannot provide complete and 
co n s i s te n t   i m p l e m e nt at i o n   d at a   f ro m   p a r t i c i p at i n g   co u nt r i e s 
from the beginning of the programme which are necessar y for 
mea su r ing  pro gress towards obj ec ti ves.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
17

30.  

31.  

M A N AgE M E N T

 the audit dealt with aspects of the day-to- day operations of the 
p ro gra m m e.   I t   a s s e s s e d   w h e t h e r   t h e   p ro ce d u re s   fo r   a gre e i n g 
on   an  an nu al   wo r k  p ro gramme  could  prov ide   th e  commiss ion 
w i t h   a   p r e l i m i n a r y   a s s u r a n c e   t h a t   t h e   p r o g r a m m e   w o u l d   b e 
implemented in accordance with the provisions in the relevant 
f r a m e w o r k   d o c u m e n t s .   t h e   a u d i t   a l s o   a s s e s s e d   w h e t h e r   t h e 
commission and the national agencies have promoted the pro -
gramme, whether the agencies have publicised the programme 
in accordance with guidance from the commission and whether 
t h e   m a i n   d i f f i c u l t y   e x p e r i e n c e d   b y   a g e n c i e s   i n   r u n n i n g   t h e 
programme is being addressed. lastly, it examined whether the 
agenc ies’ asse ssm ent o f p ro jec t appli c ati on s  was t rans pare nt, 
well- organised and followed instructions from the commission, 
and  w het he r the weak ness es ide nt if ie d  we re  followe d up.

AR R A NgE M E N T S   fO R   A P P R O V I Ng   ThE   NAT I O N Al 
AgE N C I E S ’  A N N UAl  w O Rk   P R OgR A M M E S   A R E 
A P P R O P R I AT E ,  Al ThO Ug h   ThE   CO M M I S S I O N   D O E S 
N OT   S YS T E M AT I C Al l Y   P R O V I D E  ThE   NAT I O N Al 
AgE N C I E S  wI Th   qUAlI TAT I V E   fE E Db ACk   ThE R E O N

 on the basis of standardised specifications and templates pro -
v i d e d   b y   t h e   c o m m i s s i o n ,   e a c h   n a t i o n a l   a g e n c y   p r e p a r e s   a n 
annual wor k programme, which the commission subsequently 
a p p r o v e s .   t h e   c o m m i s s i o n’s   p r i m a r y   a i m   i n   a p p r o v i n g   w o r k 
programmes is to gain preliminar y assurance that the national 
agenc y will implement the programme in accordance with the 
p rov i s i o n s   s e t   o u t   i n   t h e   re l e v a n t   f r a m e wo r k   d o c u m e n t s.   na-
t i o n a l   a u t h o r i t i e s   a l s o   a g re e   t o   t h e   a n n u a l   wo r k   p ro g r a m m e, 
t h e r e b y   c o m m i t t i n g   t h e m s e l v e s   t o   p r o v i d i n g   t h e   n a t i o n a l 
a g e n c y   w i t h   t h e   n e ce s s a r y   n a t i o n a l   m a tc h i n g   re s o u rce s.   th i s 
system works without major problems and so annual work pro -
grammes are approved within the deadlines, thereby enabling 
a continuous implementation of the programme along the lines 
agre ed.  al th ou g h  e nab ling  con solidated  wor k   program mes   to 
b e   s u b m i t t e d   b y   e a c h   n a t i o n a l   a g e n c y,   t h e   c o m m i s s i o n   h a s 
n o t   t a k e n   t h e   o p p o r t u n i t y   t o   p r o v i d e   t h e   n a t i o n a l   a g e n c i e s 
a n d   a u t h o r i t i e s   s y s t e m a t i c a l l y   w i t h   f e e d b a c k   o n   t h e   q u a l i t y 
of  the  wo r k  p ro gramme;  t his  wou ld  prov ide   a  conc i s e   vie w  o f 
the planned national implementation, something which would 
b e   u s e f u l   b o t h   fo r   t h e   c o m m i s s i o n   a n d   fo r   t h e   p a r t i c i p a t i n g 
countries because problems could be identified and cor rec ted 
in g oo d t ime.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
18

T hE   CO M M I S S I O N   A N D   PA R T I C I PAT I Ng   CO U N T R I E S 
P Ub lI C I S E   A N D   P R O M OT E  ThE   P R OgR A M M E  
S AT I Sf AC TO R Il Y

32.  

 the commission and national agencies have fulfilled their obli-
gations relating to promotion ac tivities by providing visibilit y 
and  by su p po r ting  t he diss em inat ion  an d  f ur th e r  explo it at io n 
o f   g o o d   p r a c t i c e   t h r o u g h   a   s e r i e s   o f   m e a s u r e s .   t h e   n a t i o n a l 
agencies have publicised the ‘calls for proposals’ in par ticipat-
ing   co u nt r ies   in   acco rdance  with   th e   gui dan ce  f rom  t he   com-
m i s s i o n   a n d   h ave   s u b m i t te d   re l e va nt   s u p p o r t i n g   i n fo r m at i o n 
abo ut  t he l eo nardo p rogram me. 

33.  

34.  

T hE   CO M M I S S I O N   hA S   N OT   A D D R E S S E D 
A P PlI C A N T S ’  D If fI C Ul T I E S   I N   fI N D I Ng   hO S T  
PA R T N E R S   I N   OThE R   CO U N T R I E S

 applicants had difficulties finding host par tners in other coun-
t r i e s .   t h e   c a u s e s   f o r   t h i s   i n c l u d e d   a   l a c k   o f   l a n g u a g e   s k i l l s 
and cultural differences. under leonardo, and in common with 
other lifelong  learning programme mobility schemes, national 
a g e n c i e s   a re   o nl y   re s p o n s i b l e   fo r   s e c u r i n g   pl a ce m e nt s   i n   fo r-
eign countries for applicants from their own countr y ; they have 
no role in helping applicants from other countries to find suit -
able ho s ts.

 th e   co m m i s s i o n   p rov i d e s   i n fo r m a t i o n ,   a d v i ce   a n d   a s s i s t a n ce 
to  p o tent ial   ap p lic ant s  l oo k ing  for   ho st   par tn e r s.  howe ver,  in 
2007 the commission closed down its european par tner-search 
d a t a b a s e   t h a t   w a s   d e s i g n e d   t o   f a c i l i t a t e   t h e   s e a rc h   fo r   p a r t -
ners, as it could not ensure that the data on potential par tners 
i n   t h e   d a t a b a s e   w e r e   o f   t h e   q u a l i t y   e x p e c t e d   b y   i t s   u s e r s   i n 
t h e   p a r t i c i p a t i n g   c o u n t r i e s .   t h i s   d a t a b a s e   h a s   n o t   y e t   b e e n 
rep l aced by a suit ab l e al ter nat ive.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
19

35.  

 national par tner-search databases already exist in some par tici-
pating   countr ie s,   b ut   t hes e  are  n ot  a  s ubs ti tute  for   a  europe -
w i d e   d a t a b a s e.   u s i n g   t h e   re s p e c t i ve   n a t i o n a l   d a t a b a s e s   a s   a 
p a r t n e r - s e a rc h   t o o l   l e a d s   t o   a   s i t u a t i o n   w h e re   t h o s e   s e e k i n g 
par tners are obliged to register with multiple databases so that 
th ey  can sign in and  make a searc h .

w E AkN E S S E S   I N  ThE   A S S E S S M E N T   Of 
A P PlI C AT I O N S

36.  

 the procedures for assessing applications and selecting projects 
in t he  par tic ipating  cou ntr ies are t ran s pare nt,  wel l- org an i se d 
a n d   i n   l i n e   w i t h   t h e   co m m i s s i o n’s   i n s t r u c t i o n s.   h owe ve r,   t h e 
assessments of applications showed examples of the following 
we a k n e s s e s   w h i c h   i n c re a s e   t h e   r i s k   t h at   s e l e c te d   p ro j e c t s   d o 
not m ee t th e aims set :

—  applications were accepted even though description of the 
c o n t e n t ,   a i m s   a n d   e x p e c t e d   r e s u l t s   o f   t h e   t r a i n i n g   w e r e 
i n s u f f i c i e n t   a s   a   b a s i s   f o r   j u s t i f y i n g   t h e   o u t c o m e   o f   t h e 
as ses sme nt s; 

—  a p p l i c a t i o n s   w e re   a c c e p t e d   e ve n   t h o u g h   i n fo r m a t i o n   o n 
co s t s   i n   t h e   a p p l i c a t i o n s   w a s   n o t   s u f f i c i e n t   to   j u s t i f y   t h e 
am ou nt s req ues ted;

— 

lack of comments provided by assessors to justify the marks 
given in each sec tion of the assessment for m, par ticular ly 
o n qu al itat ive m atters. 

37.  

 th e   co m m i s s i o n   d o e s   n o t   c o n d u c t   q u a l i t y   a s s u r a n c e   re v i e ws 
of projec t assessments of applications as it considers that this 
is the role of the national author it y. however, the commission 
h a d   n o t   i s s u e d   s p e c i f i c   g u i d e l i n e s   o n   h o w   t h e   n a t i o n a l   a u-
thorities should check the assessments of applications. without 
clear guidelines, there is a risk that checks by national authori-
ties could be less rigorous in some par ticipating countries than 
in o t hers.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
20

13  article 6(3)(d) of decision 
no 1720/2006/ec.

14  council regulation (ec, euratom) 
no 1605/2002 of 25 june 2002 on 

the financial regulation applicable 

to the general budget of the 

european communities, article 27 

(oj l 248, 16.9.2002, p. 1).

15  part 1 of the yearly report of the 
national agency. 

38.  

39.  

40.  

41.  

RE P O R T I Ng

 th e   co m m i s s i o n ,   i n   co o p e rat i o n   w i t h   t h e   p a r t i c i p at i n g   co u n-
tr ies,  h as  to  en sure  t he  co l l ec tion ,   an al ys i s  an d  proce ssin g  o f 
t h e   av a i l a b l e   d a t a   n e e d e d   to   m e a s u re   t h e   re s u l t s   a n d   e f fe c t s 
of  th e  p ro gramme 13.  this   ac t iv it y   comes  in   add ition   to  th e  i n-
depth impac t assessments under taken as par t of the mid-ter m 
a n d   f i n a l   e v a l u a t i o n s   o f   t h e   le o n a rd o   p r o g r a m m e.  t h e   a u d i t 
assessed whether there was a repor ting system in place based 
on  sma r t ob je c tives  which enabled  t he   commiss ion  to meas-
ure  the  re sul ts  and imp ac t  of  t he  program me aga i nst  plann ed 
per fo r m ance 14. 

T hE R E   A R E   ShO R TCO M I NgS   I N  ThE   S YS T E M   fO R  
R E P O R T I Ng   O N  ThE   R E S Ul T S   A N D   Ef fE C T S   Of   ThE  
l E O N A R D O   P R OgR A M M E

 the national agencies prepare an annual ac tivity repor t for the 
commission 15 providing information on the implementation of 
th e p ro gramme. the co ntent  of th e re por t  is  stipu late d by th e 
com miss ion  a nd  covers   matters  suc h  as   th e   n umbe r   an d   t y pe 
of p rojec t s  s upp o r ted and th eir  re sult s.

 the  co ur t  examined  t he 2 00 7 ac t ivit y re por t s of  th e s i x co un-
t r i e s   v i s i te d   a n d   fo u n d   a   n u m b e r   o f   s h o r t c o m i n g s,   d e s c r i b e d 
in  pa ragrap hs  41– 44.  thes e  shor tcoming s  reduce  th e  value   o f 
t h e   r e p o r t s   a s   a n   i n s t r u m e n t   f o r   i n f o r m i n g   t h e   c o m m i s s i o n 
and  o t her u sers  ab out t he res ult s and  im pac t  o f t he  leon ardo 
pro gra mme. 

 as   t h e   a n n u a l   wo r k   p ro gra m m e   i s   s t r u c t u re d   d i f fe re nt l y   f ro m 
t h e   a n n u a l   a c t i v i t y   re p o r t,   i t   i s   n o t   p o s s i b l e   to   m a k e   a   m e a n-
ingful comparison of results against planned per formance. the 
commission plans to adapt the annual ac tivity repor t struc ture 
in line wit h the  restr uc t ur ing of th e  wor k pro gramm e w it h e f-
fec t  fro m t he  2 01 1 wor k ing  p rogramme.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
21

42.  

43.  

44.  

 the information on results provided by national agencies is lim-
ited to fac tual data that do not allow an assessment of the im-
pac t of leonardo in ter ms of its objec tives, e.g. improvements 
i n   t e a c h i n g,   t h e   a c q u i s i t i o n   o f   s k i l l s   a n d   e c o n o m i c   i m p a c t s . 
fo r   e x a m p l e,   t h e   a n n u a l   a c t i v i t y   re p o r t   i n c l u d e s   i n fo r m a t i o n 
on the numbers of projec ts, but the information on the results 
and  o u tco mes o f the s up p o r ted ac t iv iti e s for  pa r t ic i pants  an d 
instit ut ion s is  limited.

 fur thermore, there is a lack of consistenc y in the national agen-
c i e s’  a p p r o a c h   t o   a n n u a l   a c t i v i t y   r e p o r t i n g,   w i t h   t h e   c o n s e -
q u e n c e   t h a t   t h e   f i n a l   r e p o r t s   s u b m i t t e d   t o   t h e   c o m m i s s i o n 
c o n t a i n   d i f f e r e n t   a n a l y s e s   o f   r e s u l t s ,   p r e s e n t e d   i n   d i f f e r e n t 
ways.

 the commission’s review of the annual ac tivity repor ts submit-
t e d   b y   n a t i o n a l   a g e n c i e s   h a s   fo c u s e d   m o re   o n   c o m p l e t e n e s s 
and coherence than on ensuring that the necessar y information 
w a s   r e p o r t e d   ( s e e   p a r a g r a p h   3 8 ) .   ho w e v e r,   t h e   c o m m i s s i o n 
re p o r te d   t h a t   i t   h a s   s e nt   fe e d b a c k   to   t h e   p a r t i c i p at i n g   co u n-
tr ies   af ter  the  as s essment  of  t he   20 0 8   year ly  repo r t s,  th ereby 
establishing a link bet ween the ex ante check of the wor k pro -
gramm e and t he  ex p o st as ses sment  of  th e an n ual repo r t  as o f 
20 09.

b O X   2

EX A M PlE S   Of  wE AkN E S S E S   I N  ThE  wAY   NAT I O N Al   AgE N C I E S  
P R E S E N T E D  T h E   R E S U lT S   O f  T h E   I M P l E M E N TAT I O N   O f  T h E   20 07 
w O Rk   P R OgR A M M E

•	

•	

•	

•	

lack of quantitative assessment of project results;

insufficient analysis of the way objectives were addressed;

general potential effects described without reference to actual evidence of impact;

examples of impacts taken from a study of the previous leonardo II programme.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
22

45.  

46.  

47.  

 national agencies also under take monitoring visits to projec ts, 
the aim being to suppor t the beneficiaries, gather and dissemi -
nate examples of best prac tice and establish or maintain good 
relatio ns b et we en the b eneficiar ie s  a n d th e n ati on al a ge n c y.

 t h e   c o m m i s s i o n   h a s   s e t   a   r e q u i r e m e n t 1 6  fo r   s e l e c t i n g   w h i c h 
b e n e f i c i a r i e s   s h o u l d   b e   v i s i t e d ,   i n c l u d i n g   t h e   s i z e   a n d   t y p e 
o f   t h e   b e n e f i c i a r y,   t h e i r   g e o gra p h i c a l   s p re a d   t h ro u g h o u t   t h e 
count r y  and t he l evel  o f eu grant  awarded.  howe ver, un lik e in 
the case of primar y controls, the commission has not specified 
a   m i n i m u m   n u m b e r   o f   m o n i t o r i n g   v i s i t s   w h i c h   t h e   n a t i o n a l 
agenc y sh ou ld u nder t ake.

16  2008 guide for national agencies 
implementing the lifelong learning 

programme, point 3.1.1.3.

17  council regulation (ec, euratom) 
no 1605/2002 of 25 june 2002 on 

the financial regulation applicable 

to the general budget of the 

european communities, article 27 

(oj l 248, 16.9.2002, p. 1).

 I n 2008 the  co mm issio n int roduced ‘qualit y an d  impa c t m on i-
tor ing’  visit s. the  o b jec t ive  wa s  to  obt ain  gre ater   in s ig ht  into 
th e q ua lit y o f programm e ac t ivit ie s  i n t he  par ti c ipat ing  cou n-
t r i e s.  th e s e   v i s i t s   i n c l u d e   m e e t i n g s   w i t h   n at i o n a l   a u t h o r i t i e s, 
a g e n c i e s   a n d   g r a n t   b e n e f i c i a r i e s.   a l t h o u g h   t h i s   i s   a   p o s i t i ve 
s t e p,   t h e   s c o p e   o f   t h e s e   v i s i t s   i s   n o t   s u f f i c i e n t l y   d e v e l o p e d . 
co m m i s s i o n   r e p o r t s   i s s u e d   i n   2 0 0 8   o n   t h e   o u t c o m e   o f   t h e s e 
v i s i t s   i n c l u d e d   f a c t s   a n d   d e v e l o p m e n t s   r e l a t i n g   t o   m o b i l i t y 
ac tivities, but did not contain any analysis of the ac tual effec ts 
of leonardo mobility projects on education systems, par ticipat-
ing   inst it ut io ns  or  ind ividu al s.   r epor t s  or   othe r  feedback  had 
not been sent systematically to the par ticipating countr ies for 
confir matio n o r  com ment in t he  fi r s t t wo year s.

T hE   CO M M I S S I O N   hA D   S TA R T E D   bU T   N OT 
Y E T   CO M PlE T E D  ThE   E S TAb lI ShM E N T   Of   A 
CO M P R EhE N S I V E   S YS T E M   fO R  ThE   M E A S U R E M E N T 
Of   ThE   I M PAC T   Of   l E O N A R D O   bY  ThE  ThI R D  Y E A R 
Of   ThE   P R OgR A M M E

48.  

 pro gra m m e   o b j e c t i ve s   s h o u l d   b e  ‘s ma r t ’  ( i . e.   s p e c i f i c,   m e a s-
urab l e,  achievab l e,  real is tic and  ti me ly ) 17. however, th is is not 
the case for the objec tives for the leonardo programme which 
are rather general, mak ing it difficult to measure how mobilit y 
pro jec t s can co nt r ib ute to  them  (s e e B ox  3).

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
23

b O X   3

l E O N A R D O   D A  VI N C I   SP E C IfI C   O b jE C T I V E S

•	

•	

•	

to support participants in training and further training activities in the acquisition and the use of 
knowledge, skills and qualifications to facilitate personal development, employability and participa-
tion in the european labour market;

to support improvements in quality and innovation in vocational education and training systems, 
institutions and practices;

to enhance the attractiveness of vocational education and training and mobility for employers and 
individuals and to facilitate the mobility of working trainees.

l E O N A R D O   D A  VI N C I   OP E R AT I O N Al   O b jE C T I V E S  

•	

•	

•	

•	

•	

•	

to improve the quality and to increase the volume of mobility throughout  europe of people in-
volved in initial vocational education and training and in continuing training, so as to increase the 
number of placements in enterprises to at least 80 000 per year by the end of the lifelong learning 
programme;

to improve the quality and to increase the volume of cooperation between institutions or organ-
isations providing learning opportunities, enterprises, social partners and other relevant bodies 
throughout europe;

to facilitate the development of innovative practices in the field of vocational education and training 
other than at tertiary level, and their transfer, including from one participating country to others;

to improve the transparency and recognition of qualifications and competences, including those 
acquired through non-formal and informal learning;

to encourage the learning of modern foreign languages;

to support the development of innovative Information and communication technology-based con-
tent, services, pedagogies and practice for lifelong learning.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

24

18  the lifelong learning 
programme  committee assists the 

commission with implementation 

of the programme. It comprises 

representatives of member 

states and gives opinions on or 

is consulted about measures to 

implement the programme.

49.  

50.  

51.  

52.  

 the  natio nal  ag encies  rep o r t o n th e  re sults  of  t he  pro gram me 
in their annual ac tivity repor ts. at the time of the cour t ’s audit 
t h e   co m m i s s i o n   h a d   n o t   p rov i d e d   d e t a i l e d   g u i d e l i n e s   t o   t h e 
n a t i o n a l   a g e n c i e s   o n   h o w   t o   e v a l u a t e   t h e   re s u l t s   o f   t h e   p ro -
gramm e o r how to  as sess  the impac t  of  m obi lit y  proj e c t s. 

 I n d ece mb er 2006, t he life l ong   lear nin g programm e commit-
tee 18 dec ided to est ab l is h a wor k ing  gro up wi th  th e  aim  of  as-
sessing the impac t of the whole lifelong  lear ning programme 
i n c l u d i n g   le o n a rd o   d a  vi n c i .  th e   re m i t   o f   t h e   wo r k i n g   g ro u p 
i n c l u d e d   t h e   i d e n t i f i c a t i o n   o f   a   s e t   o f   c o m m o n   i m p a c t   i n d i-
c a t o r s   t o   e v a l u a t e   t h e   s u c c e s s   o f   t h e   li f e l o n g   l e a r n i n g   pr o -
gramme against its objec tives. I n november 2009, the work ing 
g r o u p   m a d e   a   p r o p o s a l   t o   t h e   li fe l o n g   le a r n i n g   pr o g r a m m e 
committee for impac t indicators  and for a data collec tion and 
r e p o r t i n g   s y s t e m   w h i c h   w o u l d   a d d r e s s   s o m e   o f   t h e   c u r r e n t 
we a k n e s s e s.   h o we ve r,   by   j a n u a r y   2 0 1 0   t h e   p ro p o s a l   h a d   n o t 
been  app roved.

 o n ce   t h e   co m m i s s i o n   h a s   d e c i d e d   o n   t h e   i n d i c a to r s   i t   w a n t s 
to  me asu re,   o t her  changes   will  be   n ece s sar y,   for   ex ample   t he 
q u e s t i o n s   i n   t h e   e x i s t i n g   a n n u a l   a c t i v i t y   re p o r t s   w i l l   n e e d   to 
b e   re v i s e d   t o   e n s u re   t h a t   t h e y   c a n   b e   u s e d   fo r   g a t h e r i n g   t h e 
necessa r y  s t at is t ical   info r mat ion   for   e ffi ci e nt   ma n age me nt  of 
the programme. the repor ting module of the llplink It system 
wi ll ne ed to  b e adap ted to meet  n ew  d ata  co lle c ti on  n ee ds.

 th e   s e t t i n g   u p   o f   a n   i m p a c t   a s s e s s m e nt   wo r k i n g   gro u p   wa s   a 
p o s i t i v e   s t e p   t o w a r d s   c r e a t i n g   t h e   n e c e s s a r y   c o n d i t i o n s   f o r 
monitoring and measuring results and impac ts. however, three 
ye a r s   i nto   t h e   c u r re nt   le o n a rd o   p ro gra m m e,   t h e   co m m i s s i o n 
h a s   n o t   b e e n   a b l e   to   a s s e s s   to   w h at   ex te nt   t h e   p ro gra m m e   i s 
mee t ing it s ob jec tives,  n or d oe s i t h ave a  syste m in pla ce th at 
would enable it to do so in the future. even assuming that the 
lifelong  learning programme committee approves the impac t 
indicators in 2010, the data needed to assess the impac t of the 
programme would only be collec ted for the remaining stage of 
th e p rogramm e,  and it w il l  be ver y d if f ic ult  to  obt ai n  co mpar-
able data for the first three years of the programme  (2007–09), 
i.e. the period to be covered by the mid-term evaluation of the 
lifelong learning programme, the repor t on which is due to be 
sub mit ted in m arc h 2011.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
25

19  the court has previously reported 
on the audit of controls over the 

lifelong learning programme. 

see annual report of the court of 

auditors concerning the financial 

year 2008 (oj c 269, 10.11.2009, 

p. 181–185).

20  2008 guide for national agencies 
implementing the lifelong learning 

programme, point 3.8.1.

CO N T R OlS

53.  

 the   aud it  dea lt   w ith  t he  adequac y   of  th e   controls  c ar r i e d  out 
i n   re s p e c t   o f   le o n a rd o   a t   a l l   l e ve l s,   i . e.   by   n at i o n a l   a g e n c i e s, 
national authorities and the commission, within the framework 
of the lifelong  learning programme 19. the controls carried out 
we re   co n s i d e re d   to   b e   a d e q u ate   i f   t h e y   to o k   p l a ce   i n   a cco rd-
ance   wit h  t he   co mmis sio n’s  guidelin es  an d  und er   it s  super v i-
sion.

54.  

55.  

56.  

IN   M O S T   C A S E S ,  CO N T R Ol   O V E R   l E O N A R D O   bY 
NAT I O N Al   AgE N C I E S   CO M PlI E D  wI Th   C O M M I S S I O N 
gU I D A N C E

 na t i o n a l   a g e n c i e s   p e r f o r m  ‘p r i m a r y   c o n t r o l s ’  a t   b e n e f i c i a r y 
l e ve l.  th e y   a re   u n d e r t a k e n   to   gi ve   a s s u ra n ce   a s   to   t h e   re a l i t y 
and eligibilit y of the ac tivities suppor ted with eu funds and as 
to t he l eg al it y  and regu l ar it y of   th e un de r ly i ng  operati on s.

 the standards and minimum requirements for primar y controls 
are de fined  in th e co mmis sio n’s guide for  nat ional ag en c ie s 20. 
t h e   c o n t ro l s   c o n s i s t   o f   a   r a n g e   o f   a c t i v i t i e s :   a n a l y s i s   o f   f i n a l 
r e p o r t s ,   a   d e s k   c h e c k   o f   s u p p o r t i n g   m a t e r i a l   s u b m i t t e d   b y 
bene ficiaries at final repor t stage, an on-the -spot check during 
the implementation of a suppor ted ac tivit y, a post- completion 
aud it  and a systems  aud it  of reg ular   be ne f ic iar i es.

 pr i m a r y   co nt ro l s   a re   by   d e f i n i t i o n   co m p l i a n ce   co nt ro l s   w h i c h 
are n ot  de signed  to  fo cus  on the  res ult s  of  th e  mobili t y  me as-
u r e s.   a s   r e g a r d s   t h e   i m p l e m e n t a t i o n   o f   p r i m a r y   c o n t r o l s,   a l-
th o ug h nat io nal  ag enc ies gene rally  car r y  out th ei r  cont rols  in 
accordance with the commission’s guide for national agencies, 
exam pl e s of  weak nes s es were obs er ved :

—  at one national agenc y, the assessment and approval of the 
fin al rep o r t s  did not  res p ec t  the  de adlin e o f  4 5  days  f rom 
receip t  o f the rep o r t; 

—  at   a n o t h e r   nati o n a l   a g e n c y,   f i n a l   p ay m e nt s   to   b e n ef i c i a r-
ies  we re made w it ho u t t he requi re d feed back  f rom  all t h e 
mo b il it y p ar t icip ant s.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
 
26

21  article 8 of commission decision 
of 26 april 2007.

22  annual report of the court of 
auditors concerning the financial 

year 2008, chapter 9, paragraphs 

9.21 and 9.33.

57.  

58.  

A lThO Ug h   ThE   S E CO N D A R Y   CO N T R OlS   C A R R I E D 
O U T   bY   NAT I O N Al   A U ThO R I T I E S   gE N E R Al l Y 
P R O V I D E   R E A S O N Ab lE   A S S U R A N C E  ThAT  ThE  
P R I M A R Y   CO N T R OlS   A R E   Ef fE C T I V E ,  A   N U MbE R   Of 
wE AkN E S S E S  wE R E   I D E N T IfI E D

 the national authorit y is responsible for national controls over 
leo nardo.   fo r  this   pu r p o se,  th e  commiss ion  requires   it   to  es-
ta b lis h a syste m of secondar y cont rol s,  th e objec t ive of  wh ic h 
is to  p rovide  reaso nab l e assu ran ce  th at t he  system of  pr i mar y 
c o n t r o l s   o p e r a t e d   b y   t h e   n a t i o n a l   a g e n c i e s   i s   e f f e c t i v e 2 1.   I n 
most   c ase s,  t he  seco nd ar y  controls   c ar r ie d  out   by   t he   n ati on-
a l   a u t h o r i t i e s   g a v e   r e a s o n a b l e   a s s u r a n c e   t h a t   t h e   s y s t e m   o f 
p r i m a r y   co nt ro l s   i s   e f fe c t i ve.   howe ve r,   t h e   a u d i t   a l s o   s h owe d 
w e a k n e s s e s   i n   t h e   w a y   s e c o n d a r y   c o n t r o l s   w e r e   c a r r i e d   o u t : 
i n a d e q u a te   d o c u m e n t a t i o n   o f   s u p e r v i s o r y   p ro ce d u re s,   i n s u f-
f i c i e nt   s u p e r v i s i o n   o f   t h e   n at i o n a l   a g e n c y   a n d   we a k n e s s e s   i n 
t h e   f o l l o w - u p   o f   r e c o m m e n d a t i o n s   p r o p o s e d   b y   n a t i o n a l   o r 
com m issio n co ntro l  bod ies.

T hE   CO M M I S S I O N   O Nl Y   R E C E N Tl Y   P R O V I D E D 
CO M PlE M E N TA R Y   gU I D A N C E   O N  ThE   S P E C IfI C  
P R O C E D U R E S   CO V E R E D   bY  ThE   S E CO N D A R Y 
CO N T R OlS

 th e   co m m i s s i o n   c a r r i e s   o u t  ‘s ys te m   m o n i to r i n g’  v i s i t s   to   p a r-
ticipating countries. these focus on monitoring compliance by 
national agencies’ systems and procedures with lifelong  learn-
ing programme rules. national authorities have interpreted the 
meaning of ‘secondar y controls’ in different ways. as the cour t 
obser ved in its 2008 annual repor t, 22 commission guidance on 
t h e   re s p o n s i b i l i t i e s   o f   t h e   n a t i o n a l   a u t h o r i t i e s   l a c k e d   c l a r i t y 
on specific procedures. the cour t ’s audit of leonardo confirms 
t h i s   c o n c l u s i o n   a s   w e a k n e s s e s   w e r e   f o u n d   i n   t h e   s e c o n d a r y 
c o n t ro l s.  t h e   co m m i s s i o n   h a s   s u b s e q u e n t l y   t a k e n   c o r re c t i ve 
mea s ures   by  is su i ng   in s t ru c ti ons   to  th e  autho r ities   providing 
fur ther guidance on procedures covered by the secondar y con-
trol s. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
conclusIons and 
recommendatIons

27

59.  

60.  

 the commission took account of its own mandator y evaluations 
i n   d e s i g n i n g   t h e   le o n a r d o   p r o g r a m m e   b u t   d i d   n o t   e s t a b l i s h 
ac tion plans for other impor tant studies and repor ts. the com-
mis sio n est ab lish ed an ap pro pr iate proj e c t li fe c yc le  m an ag e -
m e n t   s ys te m .   howe ve r,   t h e   m a i n   It  s ys te m ,   l l p li n k ,   w a s   s t i l l 
inco mp le te by t he en d o f 2009,  th us hi n de r in g  prope r  re por t-
ing   and  re quir ing  t he  u se  o f  o ld  syste ms   at   t he   s ame   ti me.   b y 
m i d - 2 0 1 0   a   c o m m o n   r e p o r t i n g   t o o l   fo r   i m p a c t   m e a s u r e m e n t 
wa s  st il l no t co mp l ete for the  life lo ng  l ear nin g p rogram me.

 t h e   o p e r a t i o n a l   e l e m e n t s   o f   t h e   p r o g r a m m e ,   i n c l u d i n g   t h e 
pre paratio n o f the an nu al  wo r k  progra mme,  publi cit y, promo -
tion and the provision of information, were generally well man-
aged. never theless, there were weak nesses in some areas such 
as h el p ing  app l ic ant s  f in ding h ost  par t ne r s  i n  ot he r  count r i es 
an d  en sur ing  t he  qu al it y  o f  n ation al   ag en c ie s’  a ss es sm ents   of 
grant  ap pl icat io ns.

RE C O M M E N D AT I O N   1

(a)  af ter its analysis of the annual work programme, the commission 
s h o u l d   p r o v i d e   t h e   n a t i o n a l   a u t h o r i t i e s   w i t h   q u a l i t a t i v e   f e e d -
b a c k .   t h i s   f e e d b a c k   s h o u l d   p r ov i d e   a   co n c i s e   v i e w   o f   n a t i o n a l 
implementation and should highlight strengths and weak nesses, 
s o m e thin g w hich wo ul d b e us e f ul b oth f o r th e co mmissi o n an d 
th e nati o nal  au th o r i ti e s .

(b)  the commission should explore options to ensure a user-friendly 
a n d   e f f e c t i v e   p a r t n e r- s e a r c h   t o o l ,   e i t h e r   a t   e u  o r   a t   n a t i o n a l 
l eve l . 

(c)  t h e   c o m m i s s i o n   s h o u l d   c o n s i d e r   w h e t h e r   n a t i o n a l   a g e n c i e s 
sh o ul d f a cilit ate th e p la ce m e nt s of f o re ign p ar ticip ant s by p ro -
v i din g  oth e r  nati o n al   a g e n ci e s  w i th  inf o r m ati o n   ab o u t  h os tin g 
an d  inte r m e diar y o rg anis ati o ns  in  th e ir   ow n  co unt r y.

(d )  t h e   a s s e s s m e n t   o f   a p p l i c a t i o n s   c o u l d   b e   i m p r o v e d   b y   f u r t h e r 
developing the assessm ent handb o o k for evaluator s , sp e cif y ing 
i n   d e t a i l ,   f o r   e a c h   s e c t i o n   d e f i n e d   i n   t h e   as s e s sm e nt   f o r m ,   t h e 
p ur p os e o f th e  ev aluati o n an d th e m e th o ds  to a chi eve  this .

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
28

(e)   I n   o r d e r   t o   e n s u r e   c o n s i s t e n t   i n t e r p r e t a t i o n   o f   c o m m i s s i o n 
r u l e s ,   t h e   co m m is s i o n   s h o u l d   co n s i d e r   i n c l u d i n g   c h e c k s   o f   a s -
s e ssm e nt s  o f  ap p li c ati o ns  in  i t s   quali t y   an d   imp a c t   m o ni to r in g 
v isi t s to  p ar ti cip atin g  co u ntr i e s . 

(f )  t h e  co m missi o n  sh o ul d  f inalis e  th e  llplin k  ap p li c ati o n   f o r  re -
p o r tin g  o n  imp a c t m e asu re m e nt  w i th o u t f ur th e r  d e lay  in  o r d e r 
to guarante e colle c tion of complete, consis tent implementation 
dat a  f ro m a ll  p a r ti cip atin g  co untr i e s .

61.  

 a l t h o u g h   c o n t r o l   s y s t e m s   a t   c o m m i s s i o n   a n d   m e m b e r   s t a t e 
level were adequate in most cases, their repor ting systems did 
not focus sufficiently on the programme’s results, and even less 
o n   i t s   e f fe c t i ve n e s s   a n d   i m p a c t .  th e   co m m i s s i o n   h a d   s t a r te d 
but  no t   yet  co mp l eted  t he  establis h ment  of   a  compreh en s ive 
sys tem  fo r t he  m eas urem ent  of th e  i mpa c t  of  leon ardo by t he 
th ird  year  o f  the  p rogramme.   as  a   result,   th e  commiss ion  has 
not been in a position to measure the impac t of the first three 
yea rs o f l eo nardo —  a roun d half its lifespan .

RE C O M M E N D AT I O N   2

(a)  the commission should improve its ‘qualit y and impac t monitor-
ing s ys tem’ and co o rdinate it w ith th e annual ac tiv it y rep or ting 
cur re ntl y  c ar r i e d o u t by  th e nati o n al a g e n ci e s . 

(b)  t h e   c o m m i s s i o n   s h o u l d   a l i g n   t h e   s t r u c t u r e   o f   t h e   w o r k   p r o -
gr a mm e an d th e annu al re p o r t s o  as  to  e n ab l e  a co mp ar is o n  o f 
re su l t s  a g ains t  p l ann e d  p e r f o r m an ce.

(c)  t h e   co m m issi o n   sh o u l d   f in a lis e   i t s   wo r k   o n   s e t tin g   sm ar t  o b -
j e c ti ve s an d p e r f o r m an ce in di c ato r s  w i th o u t  f ur th e r  d e lay. 

(d )  I n   t h e   f u t u r e ,   t h e   co m m i s s i o n   s h o u l d   e n s u r e   t h a t   f o r   a ny   s u c-
cessor pro gramme a s ys tem for m easuring imp ac t is set up f rom 
th e o u t s e t .

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
29

62.  

 the   ove ral l   co nc l us ion  is  that  the   de si gn   an d  m an ag em ent   of 
the mobility scheme of the leonardo da vinci programme could 
lead  to   effec t ive  resu l ts. the   commiss ion  had  s tar ted  but  not 
ye t   co m p l e t e d   t h e   e s t a b l i s h m e n t   o f   a   c o m p re h e n s i ve   s y s t e m 
fo r   t h e   m e a s u r e m e n t   o f   t h e   i m p a c t   o f   le o n a r d o.   a s   a   r e s u l t , 
th e  co mmis sio n  c annot   yet  ass ess  h ow  th e  obj ec ti ves   are   be -
i n g   m e t   t h re e   ye a r s   i nto   t h e   p ro gra m m e   —   a ro u n d   h a l f   o f   i t s 
life sp an.

 this r epor t was adopted by chamber Iv , headed by m r m or ten 
l e v ys o h n ,   m e m b e r   o f   t h e   co u r t   o f   au d i t o r s,   i n   lu xe m b o u rg 
at it s m eet ing  o f 29 ju ne 2010.

Fo r  th e  Co u r t  o f  Au d ito r s

vítor m anuel da sIlva calde Ira 
Presi d e nt

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
 
 
30

A N N EX

Obj EC TIVES (bASED  ON Th E  lEgA l  bA SE  O f  Th E 
lIfEl ONg lEARNINg PROgR A M ME)

DEC IS ION No 172 0 /2006/EC Of ThE EUROPEAN 
PA RlI AMENT  AND Of ThE COUN CIl  of 
15 Novemb e r 20 06 estab lishing an  ac t ion 
pro gramme  in the  fiel d of lifelo ng learning

g E N E R Al   O b jE C T I V E S   Of   ThE   l IfEl O Ng   l E A R N I Ng 
PR OgR A M M E

•	

•	

to contribute through lifelong learning to the development 
of the community as an advanced k nowledge -based socie -
ty, with sustainable economic development, more and bet-
te r jo b s and greater s ocial cohe si on,  wh i le  en sur i n g  g ood 
p ro tec t io n o f th e environment for  f ut ure  ge n erati ons. 

in par ticular, it aims to foster interchange, cooperation and 
m o b i l i t y   b e t we e n   e d u c a t i o n   a n d   t r a i n i n g   s ys te m s   w i t h i n 
the communit y so that they become a wor ld qualit y refer-
en ce.

SP E C IfI C   Ob jE C T I V E S   Of   ThE   lIfEl O Ng   lE A R N I Ng  
PR OgR A M M E

•	

•	

•	

•	

•	

t o   c o n t r i b u t e   t o   t h e   d e v e l o p m e n t   o f   q u a l i t y   l i f e l o n g 
l e a r n i n g ,   a n d   t o   p r o m o t e   h i g h   p e r f o r m a n c e ,   i n n o v a t i o n 
and a european dimension in systems and prac tices in the 
fie ld ;

to   s u p p o r t   t h e   re a l i s at i o n   o f   a   eu ro p e a n   a re a   fo r   l i fe l o n g 
l ear ning ;

t o   h e l p   i m p r o v e   t h e   q u a l i t y,   a t t r a c t i v e n e s s   a n d   a c c e s s i-
b i l i t y   o f   t h e   o p p o r t u n i t i e s   fo r   l i fe l o n g   l e a r n i n g   av a i l a b l e 
w ith in m em b er s t ate s;

to reinforce the contr ibution of lifelong lear ning to social 
cohesion, ac tive citizenship, intercultural dialogue, gender 
eq ual it y and p ers on al  ful filment;

to help promote creativity, competitiveness, employability 
and  th e growt h o f an entrepren eu r ial  spir i t; 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

31

A N N EX

•	

•	

•	

•	

•	

•	

to   co nt r ibu te  to  inc reas ed  par tic i pati on   i n  l i felon g   le ar n-
i n g   b y   p e o p l e   o f   a l l   a g e s ,   i n c l u d i n g   t h o s e   w i t h   s p e c i a l 
needs and disadvantaged groups, regardless of their socio -
 eco no mic b ac kgroun d;

to  p rom ote l ang uage l ear n ing an d  lin g ui s ti c  di ve r si t y ;

to  s up p or t  th e devel op ment of  i n novat ive Ic t -bas ed con-
te nt, se r vices, p edago gies an d pra c ti ce for  l i felon g  l ear n-
ing ;

to reinforce the role of lifelong learning in creating a sense 
o f   e u ro p e a n   c i t i z e n s h i p   b a s e d   o n   u n d e r s t a n d i n g   a n d   re -
s p e c t   fo r   h u m a n   r i g h t s   a n d   d e m o c r a c y,   a n d   e n c o u r a g i n g 
to l era nce and res p ec t  for oth er pe ople s  a n d cult ure s;

to promote cooperation in qualit y assurance in all sec tors 
o f ed uc at ion and training  in e urope;

to   enco ura g e  t he  b es t  u se  o f  resu lts,   inn ovati ve   prod uc ts 
and processes and to exchange good prac tice in the fields 
cove re d   by   t h e   l i fe l o n g   le a r n i n g   pro gra m m e,   i n   o rd e r   to 
imp rove the  qualit y of educ ation  an d  tra i nin g.

l E O N A R D O   D A  VI N C I   SP E C IfI C   O b jE C T I V E S

•	

•	

•	

t o   s u p p o r t   p a r t i c i p a n t s   i n   t r a i n i n g   a n d   f u r t h e r   t r a i n i n g 
a c t i v i t i e s   i n   t h e   a c q u i s i t i o n   a n d   t h e   u s e   o f   k n o w l e d g e , 
sk ills and qualifications to facilitate personal development, 
e m p l o y a b i l i t y   a n d   p a r t i c i p a t i o n   i n   t h e   e u r o p e a n   l a b o u r 
mar ke t ;

to  su pp o r t  imp rovement s  in qua li t y  and  i nn ovati on i n vo -
c at i o n a l   e d u c at i o n   a n d   t ra i n i n g   s ys te m s,   i n s t i t u t i o n s   a n d 
p rac tices ;

to enhance the attrac tiveness of vocational education and 
training and mobilit y for employers and individuals and to 
fac il itate th e mob il it y of wor k in g  t rai n ees.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

32

A N N EX

l E O N A R D O   D A  VI N C I   OP E R AT I O N Al   O b jE C T I V E S  

•	

•	

•	

•	

•	

•	

to   imp rove   t he  q ual it y  an d  to  in c re ase   th e  volume   of   m o -
bility throughout europe of people involved in initial voca -
tional education and training and in continuing training, so 
as to increase placements in enter pr ises to at least 80 000 
p er ye ar by t he end  o f th e lifelo ng  l ear nin g p rogram me;

to   i m p rove   t h e   q u a l i t y   a n d   to   i n c re a s e   t h e   vo l u m e   o f   co -
o p e r a t i o n   b e t w e e n   i n s t i t u t i o n s   o r   o r g a n i s a t i o n s   p r o v i d-
ing learning oppor tunities, enterprises, social par tners and 
o th er rel evant b odies  t hroughout e urope;

t o   f a c i l i t a t e   t h e   d e v e l o p m e n t   o f   i n n o v a t i v e   p r a c t i c e s   i n 
t h e   f i e l d   o f   vo c at i o n a l   e d u c at i o n   a n d   t ra i n i n g   o t h e r   t h a n 
at ter tiar y level, and their transfer, including from one par-
t icip at ing co u ntr y to oth ers;

to  im p rove th e t rans p arenc y  an d reco gn it ion  of quali fi ca-
tions and competences, including those acquired through 
no n-for mal  and info r mal  lear n in g;

to  enco u rag e th e l ear ning of  m od er n  fore ign  l ang uag es ;

to  s up p or t  th e devel op ment of in n ovati ve  Ic t -bas ed con-
te nt, se r vices, p edago gies and  prac t ice  for  life lon g le ar n-
ing.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

3333

reply of the 
commIssIon

EXECUTIVE SUMMARY

I –II I.
t h e   c o m m i s s i o n   w e l c o m e s   t h e   o v e r -
a l l   c o n c l u s i o n   o f   t h e   c o u r t   o f   a u d i t o r ' s 
re p o r t   t h a t   t h e   d e s i g n   a n d   m a n a g e m e n t 
o f   t h e   m o b i l i t y   s c h e m e   o f   t h e   le o n a r d o 
d a  vi n c i   p ro g r a m m e   c o u l d   l e a d   t o   e f fe c -
tive results. 

I n d e e d ,   t h e   p r o g r a m m e   s h o w s   i n c r e a s-
ing demand which at present can be only 
par tly covered (less than 50 % of applica-
tions can be awarded within the available 
b u d g e t ) .   m o r e   t h a n   8 0   0 0 0   p e o p l e   w e r e 
f i n a n ce d   i n   2 0 0 9   ( 6 7   %   p e r s o n s   i n   i n i t i a l 
t ra i n i n g,   2 0   %   p e o p l e   o n   t h e   l a b o u r   m a r-
k e t ,   1 3   %   t e a c h e r s / t r a i n e r s ) .   b y   f a r   t h e 
m a j o r i t y   o f   p a r t i c i p a n t s   ( n e a r l y   9 0  % ) 
ex p re s s e d   t h e m s e l ve s   a s   h i g h l y    s at i s f i e d 
w i t h   t h e   s t a y   a b r o a d   i n   t h e i r   f i n a l 
repor t s. 

the positive impac t on par ticipants in the 
p re v i o u s   p ro gra m m e   h a s   b e e n   co n f i r m e d 
b y   t h e   i m p a c t   s t u d y   p u b l i s h e d   i n   2 0 0 7 . 
t h e   m o b i l i t y   e x p e r i e n c e s   i m p r o v e d   t h e 
s k i l l s   a n d   c o m p e t e n c e s   o f   yo u n g   p e o p l e 
a n d   p o s i t i v e l y   i n f l u e n c e d   t h e i r   e m p l o y -
abilit y, as well as their personal develop -
m e n t .   a  s t u d y   o n   t h e   i m p a c t   o f   t h e   le o -
nardo mobilit y of teachers and trainers in 
vocational education ( vetpro) is about to 
b e   f i n a l i s e d ,   a n d   t h e   i n i t i a l   r e s u l t s   c o n-
f i r m   t h e   p o s i t i ve   e f fe c t s   n o t   o n l y   o n   t h e 
p e r s o n a l   d e v e l o p m e n t   o f   p a r t i c i p a n t s 
but also on the vocational education and 
t r a i n i n g   s c h o o l s ,   t r a i n i n g   p r o v i d e r s   a n d 
systems. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

34

reply of the 
commIssIon

I V.  Six t h  ind ent
the commission is aware that the qualit y 
of secondar y controls by national author -
i t i e s   o n   n a t i o n a l   a g e n c i e s   v a r i e s,   a s   w a s 
also pointed out by the cour t in its state -
m e n t   o f   a s s u r a n c e   fo r   t h e   f i n a n c i a l   ye a r 
2 0 0 8 .   fo l l o w i n g   t h i s   e a r l i e r   o b s e r v a t i o n 
from the cour t, the commission has taken 
ac tion and developed detailed guidelines 
f o r   n a t i o n a l   a u t h o r i t i e s   i n   o r d e r   t o   p r o -
v i d e   b o t h   m e t h o d o l o g i c a l   s u p p o r t   a n d 
prac tical tools for the secondar y controls 
by   t h e   n at i o n a l   a u t h o r i t i e s.  th e s e   g u i d e -
lines were issued to the m ember states in 
d e c e m b e r   2 0 0 9   a n d   w e r e   a l s o   t h e   m a i n 
subjec t of a seminar for national author i -
t i e s   h e l d   i n   m a rc h   2 0 1 0   i n   o rd e r   t o   re m-
edy  th e wea k ness es  found.

I V.  S eventh  indent
the commission accepts that the wor k on 
p e r f o r m a n c e   i n d i c a t o r s   h a s   t o   b e   f i n a l-
ised. 

I t underlines, however, that the work with 
t h e   p ro gra m m e   co m m i t te e   o n   t h e   e s t a b -
l i s h m e n t   o f   i n d i c a to r s   s t a r te d   a l re a d y   i n 
2 0 0 7 .   fu r t h e r m o re,   b a s e d   o n   t h e   i m p a c t 
s t u d i e s   a n d   e v a l u a t i o n s   o f   t h e   p r e v i o u s 
l e o n a r d o   p r o g r a m m e   a n d   t h e   a v a i l a b l e 
q u a n t i t a t i v e   d a t a   a n d   t h e   q u a l i t a t i v e 
feedback from par ticipants in the cur rent 
program me,  t he commiss ion is con fident 
t h a t   t h e   le o n a rd o   d a  vi n c i   m o b i l i t y   p ro -
gram me is produc ing  ef fec tive re s ult s. 

t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ' s 
a c k n o w l e d g e m e n t   t h a t   a n   a p p r o p r i a t e 
p r o j e c t   l i f e   c y c l e   m a n a g e m e n t   s y s t e m 
ha s be en e stab l ished  and t hat th e ope ra-
t i o n a l   a s p e c t s   o f   t h e   p r o g r a m m e   w e r e 
generally well managed. the commission 
underlines the fac t that many of the man -
a g e m e n t   re c o m m e n d a t i o n s   a re   v a l i d   fo r 
t h e   w h o l e   li fe l o n g   le a r n i n g   pro g r a m m e 
(l l p) ,   b e c a u s e   t h e   le o n a rd o   p ro gr a m m e 
for ms an  integral  p ar t o f th e l lp.

IV.
t h e   c o m m i s s i o n   s t r e s s e s   t h a t   t h e   a u d i t 
mainly  covered  th e  years   2 00 7– 0 8 ,  wh i ch 
w e r e   t h e   f i r s t   t w o   y e a r s   o f   a   n e w   i n t e -
g r a t e d   p r o g r a m m e .   f u r t h e r m o r e ,   t h e 
p r o g r a m m i n g   a n d   r e p o r t i n g   c y c l e   w i t h 
n a t i o n a l   a g e n c i e s   c ove r s   t wo   ye a r s   f ro m 
t h e   n a t i o n a l   a g e n c y ’s   p re p a ra t i o n   o f   t h e 
d ra f t   wo r k   p ro gra m m e   to   t h e   f i n a l i s at i o n 
o f   t h e   y e a r l y   r e p o r t   a s s e s s m e n t   b y   t h e 
c o m m i s s i o n .   t h e r e f o r e ,   i m p r o v e m e n t s 
m a d e   a f t e r w a r d s ,   n o t a b l y   i n   2 0 1 0 ,   h a v e 
n o t   a l l   b e e n   t a k e n   i n t o   a c c o u n t   b y   t h e 
cou r t.

I n addition, it has to be noted that results 
o f   a c t i o n s   c o - f i n a n c e d   i n   2 0 0 7 – 0 8   a r e 
o f t e n   n o t   v i s i b l e   i m m e d i a t e l y   a f t e r   t h e 
ac tions, as mobilit y and training normally 
ta ke se veral ye ars  to s how  th eir  e ffe c t s. 

IV. f irs t ind ent
fo r   t h e   d e s i g n   o f   t h e   c u r r e n t   l e o n a r d o 
d a   v i n c i   p r o g r a m m e ,   t h e   c o m m i s s i o n 
l a u n c h e d   a   c o n s u l t a t i o n   p r o c e s s   w i t h 
n a t i o n a l   a g e n c i e s ,   n a t i o n a l   a u t h o r i t i e s 
a n d   s t a k e h o l d e r s   t o   t a k e   i n t o   a c c o u n t 
ear lier exper iences. the commission took 
i n t o   c o n s i d e r a t i o n   n o t   o n l y   t h e   f i n d i n g s 
a n d   s u g g e s t i o n s   f r o m   t h e   m a n d a t o r y 
e va l u at i o n s,   b u t   a l s o   u s e d   o t h e r   ex te r n a l 
s t u d i e s   a s   a   s o u r c e   o f   i d e a s   a n d   t o   p r o -
vide  ad dit io nal  ins ig ht. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

3535

reply of the 
commIssIon

— 

i t   h a s   a l r e a d y   d e v e l o p e d   d e t a i l e d 
g u i d e l i n e s   f o r   a s s e s s i n g   p r o j e c t   a p -
p l i c at i o n s.  th e   co m m i s s i o n   co n s i d e r s 
that the super vision of the qualit y as-
sessment of applications is par t of the 
remit   of   the  n ation al  auth or itie s’  se c -
o n d a r y   co nt ro l s   o f   n at i o n a l   a g e n c i e s ; 
n e v e r t h e l e s s ,   i t   i s   t e s t i n g   w h e t h e r   i t 
c a n   m o n i t o r   t h e s e   p r o c e d u r e s   m o r e 
c l o s e l y,   t a k i n g   d u e   a c c o u n t   o f   c o s t -
eff icien c y ;

— 

i t   i n c l u d e d   p e r fo r m a n ce   i n d i c ato r s   i n 
the set-up for the 2011 national agen -
c y work programmes, which were wel-
c o m e d   b y   t h e   n a t i o n a l   a g e n c y   d i r e c -
t o r s   a t   t h e i r   m e e t i n g   o f   2 0   m a y   2 0 1 0 
and will therefore be applied as of the 
plann in g and  re por t ing per iod 2 0 1 1 ;

—  the commission accepts that the work 
on per formance indicators needs to be 
fin alised.  I t  unde r lin es,  however,  th at 
t h e   w o r k   w i t h   t h e   p r o g r a m m e   c o m-
m i t t e e   o n   t h e   e s t a b l i s h m e n t   o f   i n d i-
c a t o r s   s t a r t e d   a l r e a d y   i n   2 0 0 7 .   t h e 
c o m m i s s i o n ' s   p r o p o s a l   w a s   a d o p t e d 
b y   t h e   c o m m i t t e e   a t   i t s   m e e t i n g   o f 
1 7   ju n e   2 0 1 0 .   on   t h i s   b a s i s ,   t h e   r e l-
e v a n t   d a t a   w i l l   b e   m a t c h e d   w i t h   t h e 
n o w   e s t a b l i s h e d   i n d i c a t o r s ;   t h e   f i r s t 
r e s u l t s   o f   t h i s   e x e r c i s e   a r e   e x p e c t e d 
b y   m i d - 2 0 1 1 .   fu r t h e r m o r e ,   b a s e d   o n 
th e impac t studies and evaluation s  of 
the previous leonardo programme and 
the available quantitative data and the 
qualitative feedback from par ticipants 
i n   t h e   c u r r e n t   p r o g r a m m e ,   t h e   c o m-
mission is confident that the leonardo 
d a  v i n c i   m o b i l i t y   p r o g r a m m e   i s   p r o -
ducing  effec tive re sult s. 

V.
t h e   c o m m i s s i o n   a c c e p t s   t h e   r e c o m-
m e n d a t i o n s   m a d e   b y   t h e   c o u r t   a n d   h a s 
a l re a d y   i m p l e m e nte d   t h e m   i n   f u l l   o r   p a r-
t i a l l y,   i n c l u d i n g :   t h e   q u a l i t a t i v e   f e e d-
back to national author ities on the year ly 
n a t i o n a l   a g e n c y   r e p o r t s ,   t h e   i n t e g r a-
t i o n   o f   t h e   y e a r l y   a c t i v i t y   r e p o r t s   i n t o 
t h e   m o n i t o r i n g   v i s i t s ,   t h e   a l i g n m e n t   o f 
t h e   a n n u a l   a c t i v i t y   re p o r t   s t r u c t u re   w i t h 
t h e   w o r k   p r o g r a m m e s ,   t h e   i n c l u s i o n   o f 
r e s u l t s - o r i e n t e d   i n d i c a t o r s   i n   n a t i o n a l 
a g e n c i e s '   w o r k   p r o g r a m m e s   a n d   t h e 
finalisatio n of l lp link .

t h e   c o m m i s s i o n   w i s h e s   t o   u n d e r l i n e 
that:

— 

— 

it has already vastly improved the sup -
por ting It system for the management 
o f   t h e   p ro g r a m m e   i n   2 0 0 9   a n d   i s   f u r-
th er  im proving   it s  rep or ting   faci li ti e s 
in 20 10;

i t   a l r e a d y   p r o v i d e s   q u a l i t a t i v e   f e e d-
b a c k   o n   t h e   ye a r l y   wo r k   p ro g r a m m e s 
o f   t h e   n a t i o n a l   a g e n c i e s ,   b u t   f o r   e f-
fe c t i v e n e s s   r e a s o n s   i t   h a s   l i m i t e d   i t s 
f e e d b a c k   t o   t h o s e   c a s e s   t h a t   r e q u i r e 
r e v i s i o n   b e c a u s e   t h e y   d o   n o t   c o r r e -
s p o n d   to   t h e   e x p e c te d   q u a l i t y   s t a n d-
ards ;

— 

it shares the cour t ’s view that suppor t 
in finding par tners is impor tant and it 
i s   e x p l o r i n g   o t h e r   o p t i o n s ,   s i n c e   t h e 
t r a d i t i o n a l   p a r t n e r - f i n d i n g   d a t a b a s e 
did no t  del iver  th e qu al it y ser vice ex-
pec ted;

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

36

reply of the 
commIssIon

s t a r t i n g   i n   2 0 0 7 ,   t h e   c o m m i s s i o n   h a s 
been ex tensively discussing with the pro-
g r a m m e   m a n a g e m e n t   c o m m i t t e e   a   s e t 
o f   p ro g r a m m e   i m p a c t   i n d i c a t o r s   a p p l i c -
able to the llp as a whole.  the committee 
e x p re s s e d  ‘n o   o p i n i o n’  o n   t wo   p ro p o s a l s 
t a b l e d   by   t h e   co m m i s s i o n   a n d   a s k e d   fo r 
fur ther analysis on cost- effec tiveness. the 
new proposal was adopted by the commit-
tee at its meeting of 17 june 2010.  on this 
b a s i s,   t h e   re l e v a n t   d a t a   w i l l   b e   m a tc h e d 
w i t h   t h e   n ow   e s t a b l i s h e d   i n d i c ato r s ;   t h e 
first  results  of  this  exercise  are  expec ted 
by mid-2011.

1 9.–2 0.
t h e   c o m m i s s i o n   m a i n t a i n s   t h a t   i t   i s 
n o t   i t s   r o l e   t o   f o l l o w   u p   w h e t h e r   t h e 
r e c o m m e n d a t i o n s   s e t   a t   n a t i o n a l   l e v e l 
h a v e   b e e n   f o l l o w e d   u p   b y   t h e   n a t i o n a l 
a u t h o r i t i e s .   ho w e v e r,   t h e   c o m m i s s i o n 
c o u l d   i n v i t e   n a t i o n a l   a u t h o r i t i e s   t o 
u n d e r t a k e   s u c h   f o l l o w - u p   a t   n a t i o n a l 
level. 

the commission emphasises that the final 
n a t i o n a l   r e p o r t s   t h a t   w e r e   p r e s e n t e d   i n 
due  time  by  the  national  author ities  had 
been submitted to the ex ternal contrac tor 
f o r   a n a l y s i s   a n d   t h e   r e s u l t s   h a v e   b e e n 
v a l i d a t e d   b y   t h e   c o m m i s s i o n .   c r o s s -
c u t t i n g   i s s u e s   h a v e   b e e n   i n c l u d e d   i n 
t h e   f i n a l   a c t i o n   p l a n   e s t a b l i s h e d   b y   t h e 
commission.

AUDIT SCOPE AND APPROACh

13.
th e   co m m i s s i o n   h i g h l i g ht s   t h e   f a c t   t h at 
2007 and 2008 were the first t wo years of 
the  lifelong  lear ning  programme  which, 
f o r   t h e   f i r s t   t i m e ,   i n t e g r a t e d   l e o n a r d o 
d a   v i n c i   a n d   t h e   f o r m e r   s o c r a t e s   p r o -
g r a m m e   i n t o   a   s i n g l e   p r o g r a m m e .   a s   a 
c o n s e q u e n c e ,   n e w   c o m m o n   p r o c e d u r e s 
h a d   to   b e   p u t   i n   p l a ce.   fu r t h e r m o re,   t h e 
p r o g r a m m i n g   a n d   r e p o r t i n g   c y c l e   w i t h 
n at i o n a l   a g e n c i e s   cove r s   t wo   ye a r s   f ro m 
the  preparation  of  the  wor k  programmes 
b y   n a t i o n a l   a g e n c i e s   t o   t h e   f i n a l i s a t i o n 
o f   t h e   ye a r l y   re p o r t   a s s e s s m e n t s.   th e re -
fore, improvements made after wards, not-
ably in 2010, have not all been taken into 
account by the c our t. 

ObSERVATIONS

16.
the  commission  under lines  the  fac t  that 
re s u l t s   o f   a c t i o n s   s u p p o r t e d   i n   2 0 0 7 – 0 8 
are often not visible immediately after the 
ac tions, as mobility and training normally 
t a k e   s e ve r a l   ye a r s   fo r   t h e i r   e f fe c t s   to   b e 
seen.

18.
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
f i n d i n g   t h a t ,   i n   d e s i g n i n g   t h e   n e w   le o -
n a r d o   p r o g r a m m e ,   t h e   r e s u l t s   o f   t h e 
m i d - t e r m   a n d   f i n a l   e v a l u a t i o n s   o f   t h e 
p re d e ce s s o r   p ro gra m m e   we re   t a k e n   i nto 
a c c o u n t   b y   m e a n s   o f   s t r u c t u r e d   a c t i o n 
plans. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

22 .
t h e   co m m i s s i o n   c o n s i d e r s   t h a t   e x t e r n a l 
s t u d i e s   w h i c h   a re   n o t   fo r m a l   e va l u at i o n s 
o f   t h e   p r o g r a m m e   d o   n o t   r e q u i r e   t h e 
drawing -up  o f ac t ion p l ans. 

t h e   c o n t e n t   o f   e x t e r n a l   s t u d i e s   s u c h   a s 
t h e   s t u d y   o n   o b s t a c l e s   t o   t r a n s n a t i o n a l 
m o b i l i t y  ‘ m ov e- it ’  a n d   t h e   i m p a c t   a n a l-
y s i s   o f   l e o n a r d o   d a   v i n c i   m o b i l i t y   o n 
y o u n g   t r a i n e e s   a n d   e m p l o y e e s ,   w h i c h 
a re   q u o te d   by   t h e   co u r t,   i s   u s e d   a s   b a c k-
g r o u n d   i n f o r m a t i o n   a n d   a s   a   s o u r c e   o f 
i d e a s   f o r   f u r t h e r   d e v e l o p m e n t   o r   a d a p -
t a t i o n   o f   t h e   p r o g r a m m e   a n d   i t s   r u l e s . 
they also provide a more detailed insight 
i n t o   c e r t a i n   a s p e c t s   o f   t h e   p r o g r a m m e 
ac tions.

b ox 1. f irs t inde nt
t h e   c o m m i s s i o n   a g r e e s   t h a t   t h e r e   i s 
s t i l l   ro o m   fo r   i m p rove m e nt   i n   t h e   i m p l e -
m e nt at i o n   o f   t h e   p ro gra m m e   i n   t h e   h o s t 
c o u n t r i e s   a n d   i n   t h e   p re p a r a t i o n   o f   p a r -
ticipants, as pointed out in the recommen -
d a t i o n   o f   t h e   i m p a c t   s t u d y   m e n t i o n e d .  
ho w e v e r,   t h e   s t u d y   a l s o   c o n c l u d e s   t h a t 
t h e   f u n d a m e n t a l   p r i n c i p l e s   a n d   m e c h a-
n i s m s   o f   t h e   p r o g r a m m e   n e i t h e r   n e e d 
n o r   o u g h t   t o   b e   c h a n g e d .   t h e   s t u d y 
c o n c l u d e s   t h a t :  ‘ t h e   l e o n a r d o   d a   v i n c i 
m o b i l i t y   p ro gra m m e   c a n   b e   d e s c r i b e d   a s 
p a r t i c u l a r l y   e f f i c i e nt ,   e f fe c t i ve   a n d   co s t-
effec tive.’

27 .
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
ove r a l l   a p p re c i a t i o n   o f   t h e   m a n a g e m e n t 
o f   t h e   p r o j e c t   l i f e   c y c l e   f o r   t h e   p r o -
gramme.

3737

reply of the 
commIssIon

a s   t h e   c o u r t   p o i n t s   o u t ,   t h e   s u p p o r t i n g 
It  s y s t e m   c o u l d   b e   i m p r o v e d .   t h e   c o m-
m i s s i o n   n o t e s   t h a t   a   t o t a l l y   n e w   I t   s y s -
t e m ,   c o v e r i n g   a   m u l t i p l i c i t y   o f   s u b p r o -
g r a m m e s   a n d   a c t i o n s ,   h a d   t o   b e   p u t   i n 
p l a ce   fo r   t h e   n e w   l l p,   o f   w h i c h   le o n a rd o 
d a  vi n c i   i s   o n l y   a   p a r t .   I n   t h e   m e a n t i m e, 
t h e   q u a l i t y   o f   i m p l e m e n t a t i o n   re p o r t i n g 
h a s   i n c r e a s e d   a n d   i s   n o w   r o b u s t .   f u r -
t h e r m o r e ,   e i g h t   e l e c t r o n i c   a p p l i c a t i o n 
fo r m s   we re   o f fe re d   fo r   t h e   2 0 1 0   c a l l,   t wo 
re p o r t   fo r m s   a re   i n   p ro d u c t i o n   a n d   t h re e 
m o r e   a r e   d u e   t o   b e   p u t   i n t o   p r o d u c t i o n 
i n   2 0 1 0 .   t h e   o v e r w h e l m i n g   m a j o r i t y   o f 
le o n a r d o   d a  vi n c i   a p p l i c a t i o n s   a r e   t h u s 
b e i n g   t r e a t e d   e l e c t r o n i c a l l y   i n   2 0 1 0 , 
t h e re by   i n c re a s i n g   e f f i c i e n c y   a n d   re d u c-
in g th e scope for  manual input  er ror s.

2 8.
t h e   c o m m i s s i o n   r e i t e r a t e s   t h a t ,   w i t h 
re g a rd   to   It  s ys te m s,   i m p o r t a nt   d e ve l o p -
m e n t s   to o k   p l a ce   i n   2 0 0 9   a n d   co n t i n u e d 
i n   2 0 1 0 .   I m p r o v e m e n t s   m a d e   a f t e r   t h e 
c o u r t ’s   a u d i t ,   n o t a b l y   i n   2 0 1 0 ,   h a v e   n o t 
all been  tak en into account by  th e  c our t.

2 9.
s i n c e   t h e   s t a r t   o f   t h e   l l p,   t h e   c o m m i s -
s i o n   h a s   c o n t i n u e d   t o   g a t h e r   q u a n t i t a-
t i v e   d a t a   a n d   q u a l i t a t i v e   f e e d b a c k   f r o m 
p a r t i c i p a nt s   o n   le o n a rd o   d a  vi n c i   m o b i l-
i t y   t h r o u g h   r a p 4l e o 1,   r e s u l t i n g   i n   a n 
uninter  rupted and reliable data ser ies for 
th e programme. 

I n   a d d i t i o n ,   t h e   c o m m i s s i o n   u n d e r l i n e s 
t h a t ,   s i n c e   t h e   e n d   o f   2 0 0 8 ,   l l p li n k   h a s 
b e e n   u s e d   fo r   t h e   p ro d u c t i o n   o f   u n i fo r m 
a n d   s t a n d a r d i s e d   f i n a n c i a l   r e p o r t s   a n d 
m a n a g e m e nt   s t at i s t i c s.  th e   co m b i n at i o n 
o f   t h e   s t a t i s t i c a l   d a t a   f ro m   r a p 4l e o   a n d 
l l p li n k   e n s u r e s   t h e   q u a l i t y   a n d   c o h e r -
ence of data for leonardo da vinci mobil-
it y.

1 

 rap4leo is an It tool that makes it possible, among other 

things, to collect quantitative data and the opinions of 

participants in leonardo da vinci mobility actions.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
38

reply of the 
commIssIon

3 2.
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
o p i n i o n   t h a t   t h e   p a r t i c i p a t i n g   c o u n t r i e s 
a n d   t h e   c o m m i s s i o n   p u b l i c i s e   a n d   p r o -
mote th e programme satis fac tor ily.

3 3.-35 .
t h e   c o m m i s s i o n   i s   a w a r e   o f   d i f f i c u l t i e s 
o n   t h e   p a r t   o f   s o m e   a p p l i c a n t s   i n   f i n d -
i n g   h o s t   p a r t n e r s   i n   o t h e r   co u nt r i e s,   a n d 
n o t e s   t h e   s u g g e s t i o n   t o   f a c i l i t a t e   t h i s 
p r o c e s s .   t h e   p r e v i o u s   p a r t n e r - s e a r c h 
d a t a b a s e   w h i c h   e x i s t e d   u n d e r   t h e   l e o -
n a r d o   d a  vi n c i   I I   p r o g r a m m e   w a s   c l o s e d 
down because it could not ensure a suffi-
c i e nt   q u a l i t y   o f   d at a   e nt r i e s   i nto   t h e   s ys-
tem. o th er  solut ion s h ave  to be explored 
a s   s u i t a b l e   a l t e r n a t i v e s .   a  p r o m i s i n g 
e x a m p l e   i s   t h e   e u r o a p p r e n t i c e s h i p   n e t -
wo r k   a n d   p l a t fo r m ,   s e t   u p   by   co m p e te n t 
b o d i e s   i n c l u d i n g   c h a m b e r s   o f   c r a f t s   o r 
co m m e rce   a n d   vo c at i o n a l   e d u c at i o n   a n d 
t ra i n i n g   p rov i d e r s,   w h i c h   s t a r te d   i n   2 0 1 0 
fo l l ow i n g   a   c a l l   fo r   p ro p o s a l s,   a n d   w h i c h 
may be better suited to address the issue 
o f   f i n d i n g   q u a l i t y   h o s t   e n t e r p r i s e s .   t h e 
co m m i s s i o n   co n s i d e r s   t h at   s u c h   a l te r n a-
t i ve s   co u l d   b e   ex p l o re d   fo r   t h e   n ex t   p ro -
gram me gen eration.

3 6.
t h e   c o m m i s s i o n   i s   p l e a s e d   t o   n o t e   t h a t 
the procedures for assessing applications 
a n d   s e l e c t i n g   p ro j e c t s   i n   t h e   p a r t i c i p a t -
i n g   c o u n t r i e s   a r e   f o u n d   t o   b e   t r a n s p a r -
e n t ,   w e l l - o r g a n i s e d   a n d   i n   l i n e   w i t h   t h e 
commiss ion’s in struc tions.

I t   s t r e s s e s   t h a t   t h e   r u l e s   o f   p r o c e d u r e 
i n   t h e   g u i d e   f o r   n a t i o n a l   a g e n c i e s   a r e 
d e s i gn e d   to   gi ve   s u f f i c i e nt   a s s u ra n ce   o n 
th e qualit y  of th e selec ted  grant s.

s t a r t i n g   i n   2 0 0 7 ,   t h e   c o m m i s s i o n   h a s 
been ex tensively discussing with the pro -
gra m m e   m a n a g e m e nt   co m m i t te e   a   s e t   o f 
p ro gra m m e   i m p a c t   i n d i c ato r s   a p p l i c a b l e 
t o   t h e   l l p  a s   a   w h o l e .   t h e   c o m m i t t e e 
e x p r e s s e d   ‘ n o   o p i n i o n’  o n   t w o   p r o p o s-
a l s   t a b l e d   b y   t h e   co m m i s s i o n   a n d   a s k e d 
f o r   f u r t h e r   a n a l y s i s   o n   c o s t - e f f e c t i v e -
n e s s .  t h e   n e w   p r o p o s a l   w a s   a d o p t e d   b y 
t h e   c o m m i t t e e   a t   i t s   m e e t i n g   o f   1 7   ju n e 
2010. on this basis, the relevant data will 
b e   m a t c h e d   w i t h   t h e   n o w   e s t a b l i s h e d 
i n d i c a t o r s ;   t h e   f i r s t   r e s u l t s   o f   t h i s   e xe r -
cise  are exp ec ted by  mid-2 01 1. 

31 .
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
f i n d i n g   t h at   a r ra n g e m e nt s   fo r   a p p rov i n g 
t h e   n a t i o n a l   a g e n c i e s’  a n n u a l   w o r k   p r o -
gramme s are ap p rop r iate. 

f o r   r e a s o n s   o f   c o s t - e f f e c t i v e n e s s ,   t h e 
c o m m i s s i o n   h a s   l i m i t e d   i t s   f e e d b a c k   t o 
n a t i o n a l   a g e n c i e s   o n   t h e i r   d r a f t   a n n u a l 
w o r k   p r o g r a m m e   t o   t h o s e   c a s e s   r e q u i r -
i n g   r e v i s i o n ,   i n   o r d e r   t o   e n s u r e   t h a t   t h e 
expec ted qua lit y  standard s  are  ach ie ved.

t h e   c o m m i s s i o n   u n d e r l i n e s ,   h o w e v e r, 
that it does provide the national author i-
t i e s   a n d   n a t i o n a l   a g e n c i e s   w i t h   d e t a i l e d 
qualitative feedback on programme man-
a g e m e nt   a n d   i m p l e m e nt at i o n   i n   i t s   e va l-
u at i o n   co n c l u s i o n s   o f   t h e   ye a r l y   n at i o n a l 
a g e n c y   re p o r t ,   i n   w h i c h   i t   a l s o   i n d i c a te s 
e x p e c t e d   r e m e d i a l   a c t i o n ,   a s   n e e d e d . 
th e   co m m i s s i o n   a l s o   c h e c k s   t h at   re m e d -
i a l   a c t i o n s   a re   i n c l u d e d   i n   t h e   fo l l o w i n g 
n a t i o n a l   a g e n c y   w o r k   p r o g r a m m e ,   t o 
ensure a continuous improvement in pro -
gramme  imp lem entat ion .

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

3939

reply of the 
commIssIon

4 4.
f r o m   t h e   2 0 0 8   a n n u a l   a c t i v i t y   r e p o r t s 
o n w a r d s ,   t h e   c o m m i s s i o n   h a s   a l r e a d y 
s e n t   e x t e n s i v e   e v a l u a t i o n   c o n c l u s i o n s 
t o   t h e   n a t i o n a l   a u t h o r i t i e s   a n d   n a t i o n a l 
a g e n c i e s .   t h e s e   e v a l u a t i o n   c o n c l u s i o n s 
p rov i d e   d e t a i l e d   fe e d b a c k   o n   t h e   q u a l i t y 
o f   t h e   p r o g r a m m e   i m p l e m e n t a t i o n   a n d 
m a n a g e m e n t   i n   t h e   c o u n t r y   c o n c e r n e d , 
a n d   a i m   t o   g u i d e   n a t i o n a l   a g e n c i e s   s o 
t h a t   t h e y   i m p r o v e   t h e i r   p e r f o r m a n c e . 
thus, a direc t link is established bet ween 
the national agenc y work programme and 
th e an nual ac tivit y  re por t.

4 6.
national agencies are indeed required, as 
the cour t notes, to under take monitor ing 
a c t i v i t i e s   to   s u p p o r t   p ro gr a m m e   b e n e f i-
c i a r i e s ,   i n   a d d i t i o n   t o   c h e c k s   o f   b e n e f i-
c i a r i e s   w h i c h   f o c u s   o n   e n s u r i n g   c o r r e c t 
u s e   o f   e u  f u n d s .   I n   a   c o n t e x t   o f   l i m i t e d 
re s o u rce s   i n   n a t i o n a l   a g e n c i e s,   a n d   co n-
s i d e r i n g   t h e   s u b s t a n t i a l   co n t ro l   re q u i re -
m e n t s,   t h e   co m m i s s i o n   d i d   n o t   co n s i d e r 
i t   a p p r o p r i a t e   t o   s e t   m i n i m u m   r e q u i r e -
m e n t s   f o r   n a t i o n a l   a g e n c i e s   f o r   m o n i-
to r i n g   v i s i t s   o f   b e n e f i c i a r i e s.  th i s   a l l ows 
n a t i o n a l   a g e n c i e s   t o   o r g a n i s e   t h e   m o n i-
to r i n g   o f   b e n e f i c i a r i e s   i n   t h e   m o s t   e f fe c -
tive and appropr iate way, tak ing national 
c o n t e x t s   a n d   c o n s t r a i n t s   i n t o   c o n s i d e r -
a t i o n ,   fo r   e x a m p l e   re p l a c i n g   m o n i t o r i n g 
v i s i t s   t o   i n d i v i d u a l   b e n e f i c i a r i e s   w i t h 
m o n i t o r i n g   m e e t i n g s   b r i n g i n g   t o g e t h e r 
groups of ben eficiar ies.

37 .
a s   e s t a b l i s h e d   b y   t h e   c o m m i s s i o n ’s 
u p d a t e d   g u i d e l i n e s   i s s u e d   i n   2 0 0 9 ,   a 
q u a l i t y   a s s u r a n c e   c h e c k   o n   t h e   a s s e s s-
m e n t s   o f   g r a n t   a p p l i c a t i o n s   s h o u l d   b e 
p a r t   o f   t h e   s e c o n d a r y   c o n t r o l s   o n   t h e 
national agencies by the national author-
i t i e s .   t h e   g r a n t   a w a r d   p r o c e d u r e   —   o f 
w h i c h   t h e   a s s e s s m e n t   o f   g r a n t   a p p l i c a-
t i o n s   i s   a n   i m p o r t a n t   e l e m e n t   —   i s   p a r t 
of the ‘key controls’ on which the national 
a u t h o r i t i e s   h ave   to   ex p re s s   t h e i r   o p i n i o n 
in their year ly  declaratio n o f assura nce.   

41 .
a  r e v i s e d   s e t u p   f o r   t h e   n a t i o n a l   a g e n c y 
wo r k   p ro gra m m e s   h a s   b e e n   d e s i gn e d   fo r 
2 0 1 1 ,   i n c l u d i n g   t a rg e t s   a n d   p e r fo r m a n ce 
i n d i c a t o r s   w h i c h   w i l l   f u r t h e r   f a c i l i t a t e 
meaning ful compar ison of results against 
p l a n n e d   p e r fo r m a n ce.  th e   n e w   a p p ro a c h 
w a s   w e l c o m e d   b y   t h e   n a t i o n a l   a g e n c y 
direc tors at their meeting of 20 m ay 2010 
a n d   w i l l   t h u s   b e   i m p l e m e n t e d   f o r   p l a n -
nin g a nd rep o r t in g as o f 20 11 .

43 .
I n   o r d e r   t o   a d d r e s s   t h e   l a c k   o f   c o n s i s t -
e n c y   i n   t h e   n a t i o n a l   a g e n c i e s’  a p p r o a c h 
to annual ac tivit y repor ting, the commis-
s i o n   h a s   p ro v i d e d   a   s i n g l e   o n l i n e   re p o r t 
fo r m   fo r   u s e   b y   a l l   n a t i o n a l   a g e n c i e s.   I n 
addition, based on the exper ience of pre -
v i o u s   y e a r s ,   t h e   c o m m i s s i o n   p r o v i d e d 
d e t a i l e d   i n s t r u c t i o n s   f o r   c o m p l e t i o n   o f 
the  year ly  re p or ts in 2009.

b ox 2
t h e   c o m m i s s i o n   u n d e r l i n e s   t h a t   t h e 
w e a k n e s s e s   f o u n d   b y   t h e   c o u r t   i n   t h e 
way in which national agencies presented 
t h e   re s u l t s   o f   t h e   i m p l e m e n t a t i o n   o f   t h e 
2007 wor k programmes refer to three out 
o f   t h e   s i x   n a t i o n a l   a g e n c i e s   e x a m i n e d 
a n d   d i d   n o t   s ys te m a t i c a l l y   o cc u r   i n   e a c h 
of t he th ree. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

40

reply of the 
commIssIon

I t   a l s o   p o i n t s   o u t   t h a t   a   l a r g e   s e t   o f 
q u a n t i t a t i v e   d a t a   a r e   a v a i l a b l e   i n   t h e 
r ap4l eo and llplink databases.  r ap4l eo 
in addition contains qualitative feedback 
f ro m   p a r t i c i p a nt s   t h at   s h ows   a   ve r y   h i g h 
d e g r e e   o f   s a t i s f a c t i o n   w i t h   t h e i r   l e o -
n ardo exper ien ces. 

4 9.
I n order to improve the quality and coher-
e n c e   o f   a n n u a l   r e p o r t i n g   b y   n a t i o n a l 
agencies, the commission provided them 
w i t h   e x t e n s i v e   g u i d e l i n e s   t h e r e o n   i n 
2 00 9.

5 0.
with regard to the development of impac t 
i n d i c a to r s   fo r   t h e   li fe l o n g   le a r n i n g   pro -
gra m m e,   t h e   l l p  co m m i t te e   a d o p te d   t h e 
re v i s e d   a p p ro a c h   p ro p o s e d   b y   t h e   co m-
miss ion  at  it s meeting  of 1 7  june  2 0 10 .

5 1.–5 2.
s i n c e   t h e   s t a r t   o f   t h e   l l p,   t h e   c o m m i s -
s i o n   c o n t i n u e d   t o   g a t h e r   q u a n t i t a t i v e 
d a t a   a n d   q u a l i t a t i v e   fe e d b a c k   f r o m   p a r-
t i c i p a n t s   o n   l e o n a r d o   d a   vi n c i   m o b i l i t y 
through r ap4l eo, resulting in an uninter -
r u p t e d   a n d   r e l i a b l e   d a t a   s e r i e s   f o r   t h e 
program me. 

I n   a d d i t i o n ,   t h e   c o m m i s s i o n   u n d e r l i n e s 
t h a t ,   s i n c e   t h e   e n d   o f   2 0 0 8 ,   l l p li n k   h a s 
b e e n   u s e d   fo r   t h e   p ro d u c t i o n   o f   u n i fo r m 
a n d   s t a n d a r d i s e d   f i n a n c i a l   r e p o r t s   a n d 
m a n a g e m e nt   s t at i s t i c s.  th e   co m b i n at i o n 
o f   t h e   s t a t i s t i c a l   d a t a   f ro m   r a p 4l e o   a n d 
llplink ensure the qualit y and coherence 
of data for l eon ardo da vin ci mobilit y.

47 .
the  concept   of  ‘qualit y  and  im p ac t   m on i-
t o r i n g   v i s i t s ’  w a s   n e w l y   i n t r o d u c e d   i n 
2 0 0 8   a n d   h a s   b e e n   g r a d u a l l y   i m p r o v e d 
s i n c e   t h e n   i n   o r d e r   t o   g a i n   f i r m   a s s u r -
a n ce   o f   t h e   e f fe c t s   o f   t h e   p ro gra m m e   o n 
t h e   b a s i s   o f   e x a m p l e s   o f   a c t u a l   p ro j e c t s. 
f e e d b a c k   h a s   b e e n   s e n t   t o   n a t i o n a l 
a g e n c i e s   o n   a   re g u l a r   b a s i s   s i n ce   au g u s t 
2 0 0 8 .   fo r   t h e   s e c o n d   ro u n d   o f   le o n a rd o 
d a   v i n c i   q u a l i t y   a n d   i m p a c t   m o n i t o r i n g 
s t a r t i n g   i n   2 0 1 0 ,   v i s i t s   w i l l   h a v e   a   m o r e 
regional  fo cu s. 

48 .
t h e   o b j e c t i v e s   o f   t h e   l e o n a r d o   p r o -
gramme are specified in the legal basis of 
t h e   li fe l o n g   le a r n i n g   pro gra m m e,   w h i c h 
h a s   b e e n   a gre e d   by   t h e   eu ro p e a n   pa r l i a -
m e n t   a n d   t h e   c o u n c i l .   t h e   c o m m i s s i o n 
i s   w o r k i n g   o n   d e v e l o p i n g   s u i t a b l e   i n d i-
c ato r s   fo r   t h e s e   o b j e c t i ve s   fo r   t h e   l l p  a s 
a   w h o l e .   t h i s   i s   a   p a r t i c u l a r l y   c o m p l e x 
exercise, given the fac t that 58 objec tives 
h a v e   b e e n   s t i p u l a t e d   i n   t h e   l e g a l   b a s i s . 
t h e   c o m m i s s i o n   a c c e p t s   t h a t   t h e   o v e r -
a l l   s y s t e m   f o r   r e p o r t i n g   o n   t h e   r e s u l t s , 
e f f e c t s   a n d   i m p a c t   b a s e d   o n   p e r f o r m -
a n c e   i n d i c a t o r s   h a s   y e t   t o   b e   f i n a l i s e d . 
h owe ve r,   at   i t s   m e e t i n g   o f   1 7   j u n e   2 0 1 0 , 
the llp committee adopted the approach 
propos ed  by t he c o mm issio n. 

t h e   c o m m i s s i o n   s t r e s s e s ,   h o w e v e r,   t h a t 
evidence from the ex ternal studies on the 
e f f e c t s   o f   l e o n a r d o   m o b i l i t y   f o r   y o u n g 
t r a i n e e s   a n d   g r a d u a t e s 2  a n d   t h e   i m p a c t 
o f   v e t p r o   m o b i l i t y   o n   i n d i v i d u a l s   a n d 
s y s t e m s 3  i n d i c a t e s   t h a t   le o n a rd o   m o b i l-
i t y   f u l f i l s   t h e   p ro gra m m e   o b j e c t i ve s   to   a 
la rge ex tent.

2  http://ec.europa.eu/education/pdf/doc218_en.pdf 

3  final report will be published soon. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

 
4141

reply of the 
commIssIon

5 6. S eco nd  ind ent
t h e   e x a m p l e   fo u n d   b y   t h e   co u r t   o f   f i n a l 
p a y m e n t s   t o   b e n e f i c i a r i e s   h a v i n g   b e e n 
made without the required feedback from 
a l l   m o b i l i t y   p a r t i c i p a n t s   i s   c o n f i n e d   t o 
one  national agen c y only. 

5 8.
t h e   c o m m i s s i o n   h a s   p r o v i d e d   g u i d a n c e 
t o   n a t i o n a l   a u t h o r i t i e s   o n   t h e   r e q u i r e d 
s e co n d a r y   co nt ro l s   s i n ce   2 0 0 8 .   ba s e d   o n 
t h e   e x p e r i e n c e   o f   t h e   f i r s t   t w o   y e a r s   o f 
t h e   p r o g r a m m e   a n d   t h e   c o m m i s s i o n ’s 
ow n   s u p e r v i s o r y   a u d i t s,   a s   we l l   a s   i n   t h e 
follow-up of the cour t ’s recommendation 
i n   t h e   co ntex t   o f   t h e   da s  2 0 0 8 ,   t h e   co m-
m i s s i o n   h a s   co m p l e m e n te d   t h e s e   g u i d e -
lines in 2009 with specific procedures for 
secon dar y  controls.

CONClUSION AND  
RECOMMENDATIONS

59.
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
conclusion that it established an appropri-
ate  projec t  lifec ycle  management  system 
for the lifelong l earning p rogramme. 

th e   co m m i s s i o n   to o k   i nto   co n s i d e rat i o n 
n o t   o n l y   t h e   f i n d i n g s   a n d   s u g g e s t i o n s 
from the mandator y evaluations, but also 
t h o s e   f r o m   e x t e r n a l   s t u d i e s .   ho w e v e r, 
the main purpose of ex ternal studies and 
repor ts is to provide the commission with 
b a c k g r o u n d   i n f o r m a t i o n   a n d   a n a l y s i s ; 
never theless,  their  status  is  substantially 
different from the mandator y evaluations 
r e q u i r e d   b y   t h e   l e g a l   b a s i s .   t h u s ,   t h e 
commission is not obliged to follow them 
up  in  the  same  for mal  way  as  mandator y 
evaluations.

s t a r t i n g   i n   2 0 0 7 ,   t h e   c o m m i s s i o n   h a s 
been ex tensively discussing with the pro -
gra m m e   m a n a g e m e nt   co m m i t te e   a   s e t   o f 
p ro gra m m e   i m p a c t   i n d i c ato r s   a p p l i c a b l e 
t o   t h e   l l p  a s   a   w h o l e .   t h e   c o m m i t t e e 
e x p r e s s e d   ‘ n o   o p i n i o n’  o n   t w o   p r o p o s-
a l s   t a b l e d   b y   t h e   co m m i s s i o n   a n d   a s k e d 
f o r   f u r t h e r   a n a l y s i s   o n   c o s t - e f f e c t i v e -
n e s s .  t h e   n e w   p r o p o s a l   w a s   a d o p t e d   b y 
t h e   c o m m i t t e e   a t   i t s   m e e t i n g   o f   1 7   ju n e 
2010. on this basis, the relevant data will 
b e   m a t c h e d   w i t h   t h e   n o w   e s t a b l i s h e d 
i n d i c a t o r s ;   t h e   f i r s t   r e s u l t s   o f   t h i s   e xe r -
cise  are exp ec ted by  mid-2 01 1.

56 .
th e   co m m i s s i o n   i s   awa re   o f   t h e   f a c t   t h at 
many national agencies have weak nesses 
i n   t h e i r   m a n a g e m e n t   o f   p r i m a r y   c h e c k s. 
t h i s   i s   p o i n t e d   o u t   f o r m a l l y   t o   t h e 
n a t i o n a l   a u t h o r i t i e s   a n d   n a t i o n a l   a g e n-
c i e s   i n   t h e   o b s e r v a t i o n s   a n d   re c o m m e n-
d a t i o n s   i s s u e d   a f t e r   t h e   a n a l y s i s   o f   t h e 
year ly national agenc y repor ts, as well as 
fo l l ow i n g   m o n i to r i n g   a n d   a u d i t   v i s i t s   o n 
the spot. as pr imar y checks are an impor-
t a n t   e l e m e n t   fo r   a   re a s o n a b l e   a s s u r a n c e 
o n   t h e   u s e   o f   e u   f u n d s,   t h e   co m m i s s i o n 
m o n i t o r s   w e a k n e s s e s   i n   t h i s   a r e a   v e r y 
c l o s e l y   a n d   s e e s   t o   i t   t h a t   a p p r o p r i -
a t e   r e m e d i a l   a c t i o n   i s   t a k e n   a t   n a t i o n a l 
level.

56 . f irs t inde nt
t h e   c o m m i s s i o n   i s   a w a r e   t h a t   i n   o n e 
n a t i o n a l   a g e n c y   v i s i t e d   b y   t h e   c o u r t 
i n   t h e   c o n t e x t   o f   t h i s   a u d i t   t h e   4 5 - d a y 
d e a d l i n e   f o r   a s s e s s m e n t   a n d   a p p r o v a l 
o f   t h e   f i n a l   re p o r t s   i s   n o t   re s p e c te d.  th e 
co m m i s s i o n   i t s e l f   h a s   m a d e   a n   o b s e r v a-
t i o n   a n d   r e c o m m e n d a t i o n   o n   t h i s   i s s u e 
fo l l ow i n g   i t s   s ys te m s   m o n i to r i n g   v i s i t   to 
the  co unt r y in  20 08 .

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

42

reply of the 
commIssIon

Recommendatio n 1 (a)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
t i o n   1 ( a ) ,   w h i c h   i s   a l r e a d y   b e i n g   i m p l e -
m e n t e d .   t h e   c o m m i s s i o n   h a s   p r o v i d e d 
n a t i o n a l   a g e n c i e s   w i t h   q u a l i t a t i ve   fe e d-
b a c k   s i n c e   t h e   2 0 0 8   a n n u a l   r e p o r t i n g . 
fu r t h e r m o r e ,   a s   t h e   c o u r t   r e c o g n i s e d , 
i t   h a s   a l s o   p ro v i d e d   fe e d b a c k   o n   a n n u a l 
wo r k   p ro g r a m m e s   fo r   t h o s e   c a s e s   w h e re 
revision was needed in order to reach the 
expec ted qualit y  stan dards.

Recommendatio n 1 (b)
t h e   c o m m i s s i o n   p a r t l y   a c c e p t s   r e c o m-
m e n d a t i o n   1 ( b ) .   t h e   c o m m i s s i o n   d i s-
c o n t i n u e d   i t s   p r e v i o u s   p a r t n e r - f i n d i n g 
d a t a b a s e   d u e   to   q u a l i t y   a s s u ra n ce   p ro b -
l e m s .   w h i l e   a l t e r n a t i v e   s o l u t i o n s   t o 
 p a r t n e r- s e a rc h   m e c h a n i s m s   a re   c u r re nt l y 
b e i n g   a n a l ys e d,   t h e   co m m i s s i o n   e m p h a -
s i s e s   t h a t   t h e   l a c k   o f   a   p a r t n e r - s e a r c h 
d a t a b a s e ,   f o r   i n s t a n c e ,   d o e s   n o t   i n t e r -
fe re   w i t h   t h e   s u cce s s f u l   i m p l e m e n t a t i o n 
o f   t h e   p ro gra m m e,   a s   t h e   d e m a n d   by   f a r 
e xce e d s   t h e   b u d g e t   ava i l a b l e   to   s u p p o r t 
leon ardo mobilit y  proje c t s.

Recommendatio n 1 (c)
t h e   co m m i s s i o n   c o n s i d e r s   t h a t   n a t i o n a l 
a g e n c i e s   s h o u l d   n o t   h a v e   a   f o r m a l   r o l e 
o f   f a c i l i t a t i n g   p l a c e m e n t s   f o r   p a r t i c i-
p a nt s   f ro m   o t h e r   co u nt r i e s   a s   t h i s   wo u l d 
entail a major shif t in responsibilities and 
i m p l i c a t i o n s   f o r   t h e   l e g a l   r e l a t i o n s h i p s 
b e t we e n   o rg a n i s at i o n s   f ro m   o t h e r   co u n-
t r i e s .   t h i s   w i l l   n o t   p r e v e n t   t h e m   f r o m 
h a v i n g   a n   e x c h a n g e   o f   s u i t a b l e   i n t e r -
med iar ies on an  in for mal basis.

a l t h o u g h   t h e   c o m m i s s i o n ’s   It  s y s t e m , 
llp l in k ,  was  sti ll  inco mp lete   at   th e  ti me 
o f   t h e   co u r t ’s   a u d i t ,   a s   i t s   d e ve l o p m e n t 
s t a r t e d   i n   2 0 0 8 ,   t h e   s y s t e m   i s   c u r r e n t l y 
stable  and  func tioning  proper ly,  and  the 
qualit y  of  data  has  been  vastly  improved 
b y   t h e   i n t ro d u c t i o n   o f   e l e c t ro n i c   fo r m s. 
a t   t h e   s a m e   t i m e ,   t h e   p a r a l l e l   u s e   o f   a 
s p e c i f i c   t o o l   f o r   l e o n a r d o   m o b i l i t y   h a s 
e n s u r e d   t h a t   n o   d a t a   l o s s   h a s   o c c u r r e d , 
and the commission has continued to col-
lec t all necessar y data on the ac tion with-
o u t   a ny   g a p s   s i n ce   t h e   p re d e ce s s o r   p ro -
gramme.

60 .
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
a p p r e c i a t i o n   o f   o p e r a t i o n a l   e l e m e n t s 
such as the yearly work programme of the 
n at i o n a l   a g e n c i e s,   i n fo r m at i o n ,   p u b l i c i t y 
an d pro mo tio n of t he p ro gramme.

t h e   c o m m i s s i o n   d i s c o n t i n u e d   i t s   p r e v i-
ous par tner-finding database due to qual-
it y assurance problems. while alter native 
s o l u t i o n s   t o   p a r t n e r - s e a rc h   m e c h a n i s m s 
a r e   c u r r e n t l y   b e i n g   a n a l y s e d ,   t h e   c o m-
m i s s i o n   h a s   e m p h a s i s e d   t h a t   t h e   l a c k   o f 
a   p a r t n e r - s e a r c h   d a t a b a s e,   fo r   e x a m p l e, 
d o e s   n o t   p re ve n t   s u c c e s s f u l   p ro g r a m m e 
i m p l e m e n t a t i o n   a s   d e m a n d   f a r   e x c e e d s 
the budget available to suppor t leonardo 
mobilit y p rojec t s.

a s   e s t a b l i s h e d   b y   t h e   c o m m i s s i o n ’s 
u p d a t e d   g u i d e l i n e s   i s s u e d   i n   2 0 0 9 ,   a 
q u a l i t y   a s s u r a n c e   c h e c k   o n   t h e   a s s e s s-
m e n t s   o f   g r a n t   a p p l i c a t i o n s   s h o u l d   b e 
p a r t   o f   t h e   s e c o n d a r y   c o n t r o l s   o n   t h e 
national agencies by the national author-
ities.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

Recom m en datio n 1( d)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
t i o n   1 ( d ) .  th e   co m m i s s i o n   i s   wo r k i n g   o n 
fur ther developing the assessment hand-
b o o k   f o r   e v a l u a t o r s   i n   c o n n e c t i o n   w i t h 
t h e   n e w   o n l i n e   a p p l i c a t i o n   a n d   r e v i s e d 
assess ment for m.

Recom m en datio n 1( e)
t h e   c o m m i s s i o n   n o t e s   w i t h   r e g a r d   t o 
r e co m m e n d at i o n   1 ( e )   t h at   i t   i s   c u r re nt l y 
r e v i s i n g   i t s   a p p r o a c h   t o   t h e   q u a l i t y   a n d 
i m p a c t   m o n i t o r i n g   v i s i t s,   b y   i n t e g r a t i n g 
a   s m a l l   s a m p l e   c h e c k   o f   q u a l i t y   a s s e s s-
m e n t s   o f   a p p l i c a t i o n s   b y   n a t i o n a l   a g e n-
c i e s.   su b s e q u e n t l y,   t h e   co m m i s s i o n   w i l l 
a n a l y s e   t h e   a d d e d   v a l u e   o f   s u c h   c h e c k s 
a n d   t h e i r   i m p a c t   o n   r e s o u r c e s   f r o m   t h e 
point of view of the cost- effec tiveness of 
control s.

Recom m en datio n 1( f )
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
tion 1( f ).

61 .
t h e   c o m m i s s i o n   w e l c o m e s   t h e   c o u r t ’s 
a s s e s s m e n t   o f   t h e   c o n t r o l   s y s t e m s   i n 
p l a ce   at   co m m i s s i o n   a n d   n at i o n a l   l e ve l s. 
t h e s e   c o n t r o l   s y s t e m s   f o c u s   o n   p r o v i d-
i n g   a s s u r a n c e   o n   t h e   u s e   o f   e u  f u n d s   o n 
a yea r l y b asis.

t h e   c o m m i s s i o n   i s   i n fo r m e d   o f   t h e   p r o -
gramme results on a regular basis via the 
y e a r l y   n a t i o n a l   a g e n c y   r e p o r t s   o n   p r o -
g r a m m e   m a n a g e m e n t   a n d   i m p l e m e n t a-
t i o n   a s   we l l   a s   v i a   i t s   q u a l i t y   a n d   i m p a c t 
mon ito r ing  visits.

4343

reply of the 
commIssIon

t h e   c o m m i s s i o n   s t r e s s e s   t h a t   i t   c o n-
t i n u e d   t o   g a t h e r   q u a n t i t a t i v e   d a t a   a n d 
q u a l i t a t i v e   f e e d b a c k   f r o m   p a r t i c i p a n t s 
o n   l e o n a r d o   d a   v i n c i   m o b i l i t y   t h r o u g h 
r a p 4l e o,   r e s u l t i n g   i n   a n   u n i n t e r r u p t e d 
a n d   re l i a b l e   d at a   s e r i e s   o n   t h e s e   a s p e c t s 
s i n c e   t h e   s t a r t   o f   t h e   p ro g r a m m e.   pe n d-
ing a decision of the programme commit-
tee  on  t he  impa c t  indic ator s,   it   i s   n ot   ye t 
c l e a r   t o   w h a t   e x t e n t   t h e   d a t a   w i l l   f u l f i l 
th e n eed of th ese in dicators.

the commission under lines that impac ts, 
h o w e v e r,   c a n   o n l y   b e   m e a s u r e d   o v e r 
t i m e.  th e   m e m b e r   st ate s   a re   ex p e c te d   to 
repor t on the first par t of the programme 
i n   t h e i r   i n t e r i m   e v a l u a t i o n   d u e   i n   2 0 1 0 . 
t h i s   i n p u t   w i l l   b e   t i m e l y   f o r   e n h a n c i n g 
the per for mance of leonardo, both in the 
c u r r e n t   p r o g r a m m e   a n d   f o r   t h e   p r e p a r -
at ion  of th e ne w programme gen erati on .

I n   a d d i t i o n ,   t h e   c o m m i s s i o n   u n d e r l i n e s 
t h a t ,   s i n c e   t h e   e n d   o f   2 0 0 8 ,   l l p li n k   h a s 
b e e n   u s e d   fo r   t h e   p ro d u c t i o n   o f   u n i fo r m 
a n d   s t a n d a r d i s e d   f i n a n c i a l   r e p o r t s   a n d 
m a n a g e m e nt   s t at i s t i c s.  th e   co m b i n at i o n 
o f   t h e   s t a t i s t i c a l   d a t a   f ro m   r a p 4l e o   a n d 
llplink ensure the qualit y and coherence 
of data for l eon ardo da vin ci mobilit y. 

s t a r t i n g   i n   2 0 0 7 ,   t h e   c o m m i s s i o n   h a s 
been ex tensively discussing with the pro -
gra m m e   m a n a g e m e nt   co m m i t te e   a   s e t   o f 
p ro gra m m e   i m p a c t   i n d i c ato r s   a p p l i c a b l e 
t o   t h e   l l p   a s   a   w h o l e .   t h e   c o m m i t t e e 
e x p r e s s e d   ‘ n o   o p i n i o n’  o n   t w o   p r o p o s-
a l s   t a b l e d   b y   t h e   co m m i s s i o n   a n d   a s k e d 
f o r   f u r t h e r   a n a l y s i s   o n   c o s t - e f f e c t i v e -
n e s s .  t h e   n e w   p r o p o s a l   w a s   a d o p t e d   b y 
t h e   c o m m i t t e e   a t   i t s   m e e t i n g   o f   1 7   ju n e 
2010. on this basis, the relevant data will 
b e   m a t c h e d   w i t h   t h e   n o w   e s t a b l i s h e d 
i n d i c a t o r s ;   t h e   f i r s t   r e s u l t s   o f   t h i s   e xe r -
cise are expec ted by mid-2 0 11. 

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

44

reply of the 
commIssIon

6 2.
t h e   c o m m i s s i o n   w e l c o m e s   t h e   o v e r -
a l l   c o n c l u s i o n   o f   t h e   c o u r t   o f   a u d i t o r ' s 
re p o r t   t h a t   t h e   d e s i g n   a n d   m a n a g e m e n t 
o f   t h e   m o b i l i t y   s c h e m e   o f   t h e   le o n a r d o 
d a   v i n c i   p r o g r a m m e   i s   i n s t r u m e n t a l   t o 
achiev in g e ffe c t ive results. 

I t   h a s   a l r e a d y   s t a r t e d   w o r k   a n d   m a d e 
p r o g r e s s   o n   s e v e r a l   p o i n t s   h i g h l i g h t e d 
b y   t h e   c o u r t ,   s u c h   a s   q u a l i t a t i v e   f e e d-
b a c k   re g a rd i n g   t h e   a n n u a l   p ro gra m m i n g 
o f   w o r k ,   t h e   a s s e s s m e n t   o f   a p p l i c a t i o n s 
an d th e improvem ent  of th e It  too l.

the commission accepts that the wor k on 
per for mance indicators needs to be final-
ised.

I t underlines, however, that the work with 
t h e   p ro gra m m e   co m m i t te e   o n   t h e   e s t a b -
l i s h m e n t   o f   i n d i c a to r s   s t a r te d   a l re a d y   i n 
2 0 0 7 .   fu r t h e r m o re,   b a s e d   o n   t h e   i m p a c t 
s t u d i e s   a n d   e v a l u a t i o n s   o f   t h e   p r e v i o u s 
l e o n a r d o   p r o g r a m m e   a n d   t h e   a v a i l a b l e 
q u a n t i t a t i v e   d a t a   a n d   t h e   q u a l i t a t i v e 
feedback from par ticipants in the cur rent 
program me,  t he commiss ion is con fident 
t h a t   t h e   le o n a rd o   d a  vi n c i   m o b i l i t y   p ro -
gram me is produc ing  ef fec tive re s ult s.

the commission, based on the above ele -
m e n t s   a s   w e l l   a s   o n   t h e   i m p a c t   s t u d i e s , 
co n s i d e r s   t h at   i t   h a s   s u f f i c i e nt   a s s u ra n ce 
on the  effe c t iven ess o f th e p ro gra mme.

Recom m en datio n 2( a)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
t i o n   2 ( a )   a n d   u n d e r l i n e s   t h a t   t h e   e v a l u-
a t i o n   c o n c l u s i o n s   o f   t h e   a n n u a l   a c t i v i t y 
r e p o r t i n g   o f   t h e   n a t i o n a l   a g e n c i e s   a r e 
duly integrated in the commission’s mon-
itor ing  visit s o f nation al  agen c ies.

Recom m en datio n 2( b)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
tion 2(b) and will align the annual ac tivit y 
repor t struc ture with that of the national 
a g e n c y   wo r k   p ro g r a m m e s   a f t e r   t h e   re v i-
s i o n   o f   t h e   w o r k   p r o g r a m m e   s t r u c t u r e 
due in 2011.

Recom m en datio n 2( c)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
t i o n   2 ( c ) .  t h e   co m m i s s i o n ' s   re v i s e d   p ro -
posal was adopted by the llp  committee 
a t   i t s   m e e t i n g   o f   1 7   ju n e   2 0 1 0 .   on   t h i s 
b a s i s,   t h e   re l e v a n t   d a t a   w i l l   b e   m a t c h e d 
w i t h   t h e   n ow   e s t a b l i s h e d   i n d i c a to r s ;   t h e 
f i r s t   re s u l t s   o f   t h i s   exe rc i s e   a re   ex p e c te d 
by mid -2011. 

Recom m en datio n 2( d)
t h e   c o m m i s s i o n   a c c e p t s   r e c o m m e n d a-
t i o n   2 ( d )   a n d   w i l l   e n d e avo u r   to   e s t a b l i s h 
a system fo r me as ur ing t he imp ac t  of  th e 
s u c c e s s o r   p r o g r a m m e   i n   a   t i m e l y   m a n -
ner.

t h e   c o m m i s s i o n   u n d e r l i n e s   t h a t   i m p a c t 
cannot  b e meas ured f rom t he o ut se t,  but 
o n l y   a f te r   a   ce r t a i n   p e r i o d   o f   p ro gra m m e 
i m p l e m e n t a t i o n .   t h i s   i s   n o r m a l l y   d o n e 
o n   t h e   b a s i s   o f   s t u d i e s   a n d   e v a l u a t i o n s . 
t h e   f a c t   t h a t   t h e   p r e s e n t   l e g a l   b a s i s   o f 
the llp contains 58 objec tives was a par -
t i c u l a r   f a c t o r   o f   c o m p l e x i t y   a n d   d i d   n o t 
f a c i l i t a t e   t h e   t a s k   o f   d e f i n i n g   s u i t a b l e 
indicato rs.

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

european court of auditors

Special Report No 4/2010 

Is the design and management of the mobility scheme of the leonardo da Vinci programme likely to lead to 

effective results?

luxembourg: publications office of the european union

2010 — 44 pp. — 21 × 29.7 cm

Isbn 978-92-9207-796-9 

doi:10.2865/76038

special r eport no 4/2010 – Is the design and management of the mobility scheme of the l eonardo da vinci programme likely to lead to effective results?

how to obtain EU publications

free publications:

•	

•	

via	EU	Bookshop	(http://bookshop.europa.eu);

	at	the	European	Union’s	representations	or	delegations.	You	can	obtain	their	contact	details	 
on the Internet (http://ec.europa.eu) or by sending a fax to +352 2929-42758.

Priced publications:

•	

via	EU	Bookshop	(http://bookshop.europa.eu).

Priced subscriptions (e.g. annual series of the Official Journal of the European Union and 
reports of cases before the Court of justice of the European Union):

•	

	via	one	of	the	sales	agents	of	the	Publications	Office	of	the	European	Union	 

(http://publications.europa.eu/others/agents/index_en.htm).

q
j
-
a
b
-
1
0
-
0
0
5
-
e
n

-
c

the mobIlIty projects of the leonardo da vIncI  programme 

enable  organIsatIons  Involved  In vocatIonal  educatIon 

and traInIng to  send  partIcIpants to  another  european 

country,  thus  gIvIng  them  the  chance  to  Improve  theIr 

competences, knowledge and skIlls. thIs report  focuses 

on the desIgn of the programme, Its management system, 

programmIng, operatIonal elements, controls, reportIng 

and  measurement of achIevement of objec tIves, results 

a n d   I m pac t s .  t h e   r e p o r t   co n ta I n s   r e co m m e n dat I o n s 

whIch are aImed at helpIng the commIssIon to Improve the 

management of the programme.

european court of audItors