SEC(2005) 1252

COMMISSION STAFF WORKING PAPER

Evaluation of the European Year of People withDisabilities{COM(2005) 486 final}

Commission Department: DG Employment and Social Affairs, Unit Integration of People with Disabilities:

Evaluation of the European Year of People with Disabilities

Synthesis

October 2004

Contents

1. Aim of the evaluation and methodology used 1

1.1 Aims of the evaluation 1

1.2 Evaluation methodology 2

2. Description of the European Year of People with Disabilities 3

2.1 Objectives of the intervention 3

2.2 Key structures 3

2.2.1 Actions and measures and financial input 4

3. Baseline situation in the participant countries and at European level 6

3.1 Baseline situation – EU level 6

3.2 Baseline situation at national level 6

3.3 National priorities 7

4. Main evidence and findings 8

4.1 Evaluation Question No. 1: 8

4.2 Evaluation Question No. 2: 8

4.3 Evaluation Question No. 3: 10

4.4 Evaluation Question No. 4: 12

4.5 Evaluation Question No. 5: 16

4.6 Evaluation Question No. 6: 17

4.7 Evaluation Question No. 7: 21

4.8 Evaluation Question No. 8: 22

4.9 Evaluation Question No. 9: 22

5. Conclusions and recommendations 25

Aim of the evaluation and methodology used

Aims of the evaluation

This report presents the findings of the final evaluation of the “European Year of People with Disabilities 2003” (EYPD). The evaluation was commissioned by the European Commission, DG Employment and Social Affairs, Unit ‘Integration of People with Disabilities’. Ramboll Management has prepared this report and is responsible for the views presented in it. The duration of the contract has been from August 2003 to October 2004.

The evaluation was carried out in accordance with Article 13 of the Council Decision (2001/903/EC) establishing the legal framework for the intervention.

The evaluation’s terms of reference state that the scope of the evaluation is to analyse the European added value in accordance with the following criteria:

- “Analysis of the relevance: to what extent are the EYPD’s objectives pertinent in relation to the evolving needs at both national and European level? Consistency with the subsidiarity principle.

- The extent to which the EYPD’s expected outputs, results and impacts have been realised (specific and general objectives), including unforeseen results

- Evaluation of effectiveness: what has been achieved regarding what has been foreseen and how have the EYPD’s outputs contributed in promoting exchange of good practice at EU and national level and achieving its specific and general objectives (in particular the equal rights approach, promotion of structures and awareness raising).

- Evaluation of the entire implementation process and delivery mechanisms: implement the EYPD decentralised approach, mobilisation and participation from the relevant actors (including corporate sector), promotion of new partnerships, empowerment of the persons with disabilities, complementarity of the interventions at national, regional, and local level. Relevance and effectiveness of the preparatory phase in 2002, incidence on the success of the year in 2003. Means put in place at community level to insure the EU information campaign, the dissemination and the transfer of practices and their mainstreaming. Relevance of the choice of delivery mechanisms and possible alternative.

- Role of the different actors regarding the effectiveness in achieving the objectives, role of the Commission services, Committee support structures, national, regional and local authorities and other organisations, NGOs representatives of people with disabilities and/or their families, EU structures financed in the field of specific disability groups.

- Evaluation of efficiency: cost-benefit analysis of the choice of Community methods of action, in particular the EU communication campaign, the EU events. Repartition of the funding in different strands and objectives and the co-financing rate. Relationship between the objectives achieved and the resources used (financial and human).

- Evaluation of the synergy between activities at national and European level against the objectives of the EYPD.

- Evaluation of complementarity with other Community instruments and programmes such as programmes to combat Anti-discrimination, Social exclusion, Gender equality, EQUAL initiative and Youth programme

- Evaluation of EYPD’s impacts reached compared with the needs of the target population(s). To what extent can the positive changes be expected to last after the EYPD has been terminated? What is the potential impact on the targeted populations?

Evaluation methodology

Overall, the evaluation design was structured in such a way that considerable emphasis was placed on the levels of the implementing bodies (the Commission and the responsible national authorities), and to a lesser extent on the individual project level. This was done on the basis of the intervention logic of the Year. We also took the view that more aggregate data could be obtained by focusing and deploying our efforts in this way.

The evaluation was based on an analysis of the key Commission documents, interviews with representatives of the Commission, the Disability Advisory Committee and the relevant national authorities. The purpose of the interviews was to confirm the intervention logic of the Year, refine the proposed methodology, and establish a baseline for the intervention.

We subsequently set out to devise an information system that consolidated key information about relevant projects co-funded by the European Community and the actions taking place at Community and national level.

Case studies were undertaken in selected countries and at Community level. At the national level, case studies were undertaken in Austria, Finland, France, Iceland, Ireland, the Netherlands, Spain and the UK. As a component of these case studies, interviews were conducted with representatives of the national authorities, disability platforms, project evaluators and other stakeholders. The case studies at Community level focused on the March, the Corporate Partnership Programme and the information campaign. For these case studies, interviews were undertaken with representatives of inter alia the European Commission, EDF, EASPD, corporate partners, and the public relations agent organising the campaign. The case study data was intended to provide qualitative data and explanations for the quantitative data collected.

Two Internet-based impact surveys were carried out among project operators and all members of the DAC, the heads of the NCBs, and representatives of the High Level Group on Disability. The surveys were designed to aggregate the outcome data for the intervention at project level, as well as to aggregate the data for each country.

Finally, desk research was carried out on the interim and final reports submitted by the Community-wide actions and transnational projects. In addition, national evaluation reports were submitted from Ireland and Sweden and fed into the study.

Description of the European Year of People with Disabilities

Objectives of the intervention

The European Year of People with Disabilities (EYPD) aimed to drive progress towards the achievement of equal rights for people with disabilities by raising awareness at all levels and involving key stakeholders, in particular people who themselves had disabilities plus the organisations representing them.

Specifically, the Council Decision to establish the European Year sets out the following eight objectives:

1. To raise awareness of the rights of people with disabilities to protection against discrimination, and to full and equal enjoyment of their rights;

2. To encourage reflection on and discussion of the measures needed to promote equal opportunities for people with disabilities in Europe;

3. To promote the exchange of good practice and effective strategies devised at local, national and European level;

4. To reinforce co-operation among all the parties involved, namely government, the social partners, NGOs, the social services, the private sector, communities, voluntary sector groups, and people with disabilities plus their families;

5. To improve communication regarding disability, and to promote a positive image of people with disabilities;

6. To raise awareness of the heterogeneity of people with disabilities, and of the various kind of disability;

7. To raise awareness of the multiple discrimination confronting people with disabilities;

8. To pay special attention to awareness of the right of children and young people with disabilities to equality in education, so as to encourage and support their full integration in society and to promote the development of European co-operation between those professionally involved in the education of children and young people with disabilities, in order to improve the integration of pupils and students with special needs in ordinary or specialised establishments and in national and European exchange programmes.

While there are eight different objectives stated for the EYPD, the three last may logically be considered as sub-objectives to the objective of ‘raising awareness of the rights of people with disabilities to protection against discrimination, and to full and equal enjoyment of their rights’ (A). Accordingly, these three objectives logically belong to a lower hierarchical level. This is reflected in the analysis of the results of the Year, where contributions to these objectives are considered as contributions to the overall objective of raising awareness.

Key structures

The Council Decision establishing the Year gives the overall responsibility for implementation, management and organisation of the Year to the Commission, supported by an Advisory Committee. However, a central feature of the EYPD was that at the national level, most of these functions were decentralised to National Co-ordinating Bodies (NCBs) in the Member States.

The NCBs were responsible for establishing national priorities for the Year, and for the overall co-ordination and implementation at national level. This included inter alia selecting the actions that would receive support from the global grant that was allocated for administration by the Member States.

The Council Decision required the Member States to ensure that the NCBs were representative of a range of organisations working on behalf of people with disabilities and other relevant stakeholders. In most countries a wide variety of stakeholders were accordingly involved in the NCB, such as national, regional and local authorities, disability organisations, social partners, NGOs and media representatives.

At the European level, the Commission was assisted in its implementation by an advisory committee (DAC) composed of one representative from each of the participating countries, and with Norway and Iceland holding observer status. This committee was consulted on all key decisions about the actions and implementation of the Year at EU level, and also worked to co-ordinate the activities taking place at national level.

An additional aim of the DAC was to bring together the Member States and the Commission with representatives of the disability movement in the form of the European Disability Forum (EDF) and the European Association of Service Providers for People with Disabilities (EASPD). Both organisations therefore participated in the informal meetings of the DAC that were held in advance of the formal meetings.

Additionally, the EDF was a member of the steering group, and was heavily involved in the preparation and co-ordination of the Year, including defining and preparing the content of the launch, the closing and Community-wide events, and co-operating with the selected PR company concerning the strategy and implementation of the information and promotional campaign that was undertaken.

Actions and measures and financial input

The European Year had a budget of 12 MEUR, which was further increased by 780,000 EUR following an amendment by the European Parliament. 7.4 MEUR was passed to the Member States as global grants and for national launch events, while the remaining budget was allocated for a number of Community-wide activities.

The Member States were required to spend at least half of the global grant of 6.7 MEUR on co-financing projects at national, regional and local level, while the remainder was permitted to be used for the co-financing of preparatory work (up to a maximum of 10%), as well as for contracted activities in support of the objectives of the Year, such as information campaigns.

In total, more than 1,100 projects were co-financed via the global grants throughout the Member States, including such diverse actions as conferences, sports events, studies, cultural events and events designed to empower people with disabilities.

In addition to the projects and activities co-financed by the global grant, a very large number of events were organised in celebration of the Year that sought to promote the rights of people with disabilities and to raise awareness of their situation.

At Community level, the budget was used to finance actions such as the overall EU information campaign, including a ‘People’s March’ which made its way through the Member States on an EYPD bus, plus information campaigns by European NGOs, opening and closing ceremonies, a study of the situation of people with disabilities in institutions, transnational projects, and the European Parliament of Disabled Persons. Table 2-1 shows the overall distribution of the budget.

Table 2-1 Allocation of Community budget

Activity | Amount (EUR) |

2002 |

EU Information and Promotional Campaign | 2,399,781 |

National Launch events | 685,288 |

Opening ceremony in Greece | 300,000 |

Media Event | 227,747 |

NGO information campaigns | 336,728 |

European Day of People with Disabilities postcards | 2,550 |

2003 |

Global grants | 6,690,000 |

Evaluation of the European Year | 200,000 |

Closing ceremony in Italy | 400,000 |

Eurobarometer survey | 100,000 |

Study on the situation of people with disabilities in institutions | 450,000 |

Other activities, including conferences | 332,000 |

Parliament of Disabled Persons | 150,000 |

Transnational Projects | 630,000 |

TOTAL | 12,904,094 |

Source: EYPD-20, provisional data

Baseline situation in the participant countries and at European level

Baseline situation – EU level

The proposal for a council decision broadly describes the baseline situation of people with disabilities as one that is characterised by barriers in all areas of life, e.g. in employment, education and access to physical surroundings, as well as to technologies which could help overcome them (e.g. the Internet). One of the consequences of these barriers is that only 25% of people with a severe disability, and 47% of people with a non-severe disability, are in employment or self-employment, compared to 66% of the general population (according to the European Community Household Panel (ECHP) data, 1996).

On the whole, the baseline situation was therefore characterised by a lack of knowledge concerning the potentials of people with disabilities, and as such implied the need for a change of attitudes through the raising of awareness among both the general public and decision makers.

According to the Proposal for a Council Decision on the European Year of People with Disabilities 2003. COM(2001) 271 final, both public support and political will are decisive for the improvement of disability policies and hence for the lives of people with disabilities, and working towards this aim represents a great challenge.

The baseline challenges at EU level can therefore be described as:

- Building networks and exchanging experiences

- Promoting the rights-based approach in policy making

- Removing barriers in employment and education

- Improving access to physical surroundings

- Improving access to technologies that promote full citizenship

- Combating negative stereotypes and discrimination by raising awareness.

Baseline situation at n ational level

The baseline situation and challenges were also addressed in the interviews with representatives of the NCB and the DAC. It became clear from these interviews that some of the Member States’ approaches differed concerning how they tackled disability issues, as did their assessment of the baseline challenges.

Among many of the countries, access is broadly considered as being the main challenge. This includes, but is not limited to, access to employment, education, leisure time activities, buildings and transport. Also, the issue of equal opportunities and non-discrimination, which is clearly connected to access, was mentioned by a number of representatives. However, in some countries the challenges seem initially to be connected with such things as the availability of more basic financial and practical support for people with disabilities.

In this connection it is important to underline that the baseline situation and challenges existing in a particular Member State must clearly have an effect on the outcome of the EYPD and the prospects for reaching the stated objectives of that Member State. Where the development and availability of basic services or financial support is seen as the central challenge, the overall impact of the EYPD in terms of such matters as raising awareness of the rights of people with disabilities may be weakened. In addition, the extent to which the various stakeholders uphold the values of the rights-based approach may also have an impact.

While the different baseline challenges and situations of the Member States would be likely to affect the EYPD’s prospects of attaining its stated objectives in each Member State, these different approaches could favour the argument for the adoption of a decentralised approach by the EYPD, as has been the case in practice.

National priorities

The pertinence between the national priorities of the participant countries and the objectives of the Year was established by logically comparing the objectives.

As laid down in the Council Decision, each participating country had to develop its own national priorities for the Year. Further analysis of the national priorities reveals the differences concerning their degree of detail and whether they focused on specific policy areas such as employment and education, or on more general approaches such as equal rights and awareness in general. These differences make it difficult to compare the priorities of the countries directly with each other, and to some degree they also make it difficult to relate the stated priorities to the overall objectives of the EYPD as set out in the Council Decision.

In most of the participating countries, the national priorities related to several of the EYPD objectives. The least degree of relevance occurred in relation to EYPD objective C concerning the promotion of the exchange of good practice and strategies. Only for Italy and France were the national priorities found to be directly pertinent to this objective. On the other hand, it should be kept in mind that approximately one in five of the Commission-initiated activities were related to Objective C, and so in this sense the actions undertaken at European and national levels can be said to have been complementary.

Main evidence and findings

This section presents the main evidence and findings of the evaluation. The evidence and findings are structured around the evaluation questions presented in Section 1.1. For a more detailed presentation of the evidence that was collated, we refer to the evaluation report itself.

Evaluation Question No. 1:

Analysis of the relevance: to what extent are the EYPD’s objectives pertinent in relation to the evolving needs at both national and European level? Consistency with the subsidiarity principle?

The evaluator noted that of the eight different objectives stated for the EYPD, the last three can logically be regarded as sub-objectives to the objective of “raising awareness of the rights of people with disabilities to protection against discrimination, and to full and equal enjoyment of their rights”. Accordingly, these three objectives, which also cover different forms of awareness-raising, logically belong at a lower hierarchical level. This is reflected in the analysis of the results of the Year, where the contributions to these objectives are treated as contributions to the overall objective of raising awareness.

The specific objectives of the intervention cannot be considered particularly operational, and tangible expected outcomes cannot be discerned. Therefore, a relevant criticism could be that the objectives’ generic character compromises their operational attainability. However, the generic character of the objectives must be viewed in the context of the decentralised delivery mechanism of the Year. Accordingly, the national priorities established in each country can be said to have translated the generic objectives into more tangible expected results. The evaluator considers this organisational set-up to have been logically coherent with the subsidiarity principle, and one that allowed the participating countries to target those issues deemed to be the most pertinent in each case, and hence that translated the objectives of the Year into more operational entities that were relevant to the national problems.

A subsequent mapping exercise of the national priorities showed that all the participating countries had formulated specific national priorities. In most of the participating countries, the national objectives pertained to several of the EYPD objectives, which indicates a compatibility of the programming at the national level with the stated objectives of the overall intervention.

Evaluation Question No. 2:

The extent to which the EYPD’s expected outputs, results and impacts have been realised (specific and general objectives), including unforeseen results.

The evaluation question partially overlaps with the subsequent evaluation question. In accordance with the proposition, we briefly describe the kinds of outputs produced at Community and project level.

Community level

A number of tangible outputs were produced at the Community-wide level. The table below outlines the planned output. The evaluation also showed that at the time of writing, the outputs that had been planned had in fact been carried out.

Table 4-1 Kinds of Community-wide actions

Community-wide actions | Type of projects | Number |

1. Meetings and events | Opening and closing event of the EYPD[1] Congress on Media and Disability European Parliament of People with Disabilities | 4 |

2. Information and promotional campaigns | Information campaigns run by European non-governmental organisations[2] | 5 |

Transnational projects[3] | 6 |

Information and promotional campaign run by Ogilvy/EurO&M | 1 |

3. Other actions | Included in Society research project Eurobarometer Survey Evaluation of the EYPD | 3 |

Total | 19 |

National level project output

The table below provides an overview of the kinds of output by country.

Table 4-2 Kind of output by country[4]

Media and communication products | Events | Research | Training/ Advocacy | Total |

Austria | 6 | 8 | 0 | 8 | 22 |

Belgium | 44 | 34 | 2 | 0 | 80 |

Denmark | 22 | 7 | 0 | 1 | 30 |

Finland | 0 | 0 | 0 | 0 | 0 |

France | 60 | 20 | 3 | 25 | 108 |

Germany | 0 | 0 | 0 | 0 | 0 |

Greece | 9 | 7 | 0 | 0 | 16 |

Iceland | 0 | 0 | 0 | 0 | 0 |

Ireland | 23 | 6 | 4 | 12 | 45 |

Italy | 6 | 11 | 0 | 1 | 18 |

Luxembourg | 6 | 3 | 0 | 0 | 9 |

Norway | 0 | 0 | 0 | 0 | 0 |

Portugal | 10 | 19 | 3 | 2 | 34 |

Spain | 9 | 6 | 5 | 10 | 30 |

Sweden | 30 | 11 | 0 | 6 | 47 |

The Netherlands | 65 | 64 | 1 | 27 | 157 |

UK | 44 | 43 | 6 | 77 | 170 |

Total | 334 | 239 | 24 | 169 | 766 |

The table shows that only a few studies and surveys (‘Research’) were produced by the project operators. The majority of the output consisted of different kinds of media products and events. However, in some countries training and advocacy played a very significant role.

The table below indicates that the planned project activities were in fact carried out as planned. This evidence was further corroborated by interview evidence from case studies in eight countries.

Table 4-3 Attainment of project output

To what extent were the planned project activities carried out? | Respondents | Per cent |

To a high extent | 241 | 80.6% |

To some extent | 50 | 16.7% |

To a limited extent | 5 | 1.7% |

Not at all | 0 | 0.0% |

Don’t know | 3 | 1.0% |

Total | 299 | 100.0% |

N = 299

Source: Ramboll Management survey among project operators

Evaluation Question No. 3:

Evaluation of effectiveness: what has been achieved regarding what has been foreseen and how have the EYPD’s output contributed in promoting exchange of good practice at EU and national level and achieving it specific and general objectives (in particular the equal rights approach, promotion of structures and awareness raising)?

Community level

At Community level, the ‘Equal opportunities for people with disabilities’ Action Plan is arguably the main tangible outcome of the Year. As the plan was previously announced as being a primary contribution of the Commission to the EYPD, there is clear consistency in this respect between what was envisaged and what was actually achieved at Community level. The same conclusion must be drawn regarding the establishment of the Expert Group on Accessibility which reported towards the end of 2003, and whose recommendations were taken into account in the Action Plan.

The Commission’s Action Plan was a new initiative adopted with a view to steering the follow-up to the EYPD. The plan should therefore be regarded as an outcome of the Year. Obviously, it is difficult to foresee what would have happened if the Year had not taken place – in other words, to imagine the counterfactual situation. Some of the initiatives contained in the Action Plan would probably have been initiated even without it , a point that was also recognised by the Commission, but the majority of the initiatives and the total momentum resulting from the plan are unlikely to have been achieved without the plan and thus without the existence of the Year.

Other important results at Community level were the resolutions adopted by the Council and the European Parliament, and the actions of the social partners at European level. The number of disability-related resolutions from the Council increased considerably in 2003 as compared to previous years, which indicates a clear influence of the EYPD. Whereas on their own these may only have indirect implications for the situation of people with disabilities, they do signal a political commitment towards the disability area that may lead to more tangible results in the future, and to a more general mainstreaming of disability issues into decision making.

It is difficult to assess the extent of the mainstreaming of the disability issue into all the relevant events that were intended to take place during 2003, as there is no yardstick against which to make comparisons. However, several activities did in fact integrate the disability issue during the Year. From our interviews with Commission officials, it would appear that the Disability Inter Service Group has been an important forum for ensuring that mainstreaming took place.

National level

The different data sources taken together indicate that the EYPD did in fact produce a number of strong results at the Member State level. All sources indicated that the Year led to a considerable degree of awareness-raising concerning disability issues, and that the Year also clearly contributed to other results such as networking and the involvement of different actors and the exchange and development of good practice, consistent with what had been envisaged.

As far as the specific objectives of the Year are concerned, we found that the greatest degree of achievement occurred among Objectives A, F, G, and H concerning awareness raising, in particular according to the assessments of stakeholders and the evidence from the allocation of financial resources.

The activities undertaken as part of the Year played a strong role in the attainment of the EYPD’s results at national level. As far as the projects were concerned, their contributions to the overall results of the Year derived from the high degree of attainment of their project aims (as seen in Table 4-3 above). In several cases the projects had the side effect of strengthening the organisational capacity of the project operators and the pursuit of their mission statements.

People with disabilities and disability organisations were heavily involved both in the planning and the running of the projects. As the majority of projects were carried out in partnerships involving two or more organisations, they clearly contributed to networking and the involvement of people with disabilities and their organisations.

The Community-wide activities also contributed to the overall results achieved in the Member States. The Bus helped to raise awareness of the campaign, and also created local community involvement via events that were undertaken during its visits to towns and cities throughout the Member States. The Bus was not successful in all instances, but appeared to achieve particular success when there was a connection between its activities and the local events taking place when it visited. A similar tendency characterised the remainder of the Community-wide actions.

Rights-based approach

It was a central feature of the EYPD that it focused on the rights of people with disabilities and the possibility of achieving equal opportunities for them, rather than focusing on their functional limitations. This was an intrinsic element of the overall campaign as well as in the individual project activities undertaken during the Year.

The aim of promoting the rights-based approach is clearly reflected in the objectives of the Year, and is also reflected in the fact that the Year was conducted as one of people with disabilities rather than one for people with disabilities, by including people with disabilities and their organisations in the planning and implementation of the activities at both EU and national level. The results showed that a high degree of involvement of people with disabilities was actually achieved, both in the individual projects and activities and in relation to the programming at EU and national level.

As has been concluded above, the Year achieved some good results in relation to awareness-raising. By raising awareness of the right of people with disabilities to protection against discrimination and to the full and equal enjoyment of their rights, it must be concluded that the Year did in fact help to promote the overall rights-based approach, although the wider implications are so far difficult to assess.

Additionally, the rights-based approach is reflected in a number of the policy and legal initiatives that were launched during the Year. Since the EYPD seems to have contributed positively to a number of these initiatives by functioning as a lever for such actions, the Year has also spurred the practical application of this approach.

At the EU level, the Action Plan ‘Equal opportunities for people with disabilities’ mentioned previously is an important instrument in this respect. During the action plan’s first phase this will especially be so in relation to employment, as a key rationale for the Plan is to contribute to making the right to work equally meaningful for people with disabilities.

Overall, we conclude that the EYPD has in fact contributed to both the promotion and application of the rights-based approach. Its full achievements in this regard can only be assessed in the longer term, but in the shorter term the Year’s success should be considered to be its ability to build momentum for changes and initiatives that will take the rights-based approach forward.

Good practice

The exchange of good practice figured among the objectives of the Year (Objective C), and a number of projects focused directly on this issue. The financial trail analysis showed that at the national level, approximately 15% of the global grant was allocated to this objective.

The exchange of good practice mainly occurred through three types of instruments/forums, namely the DAC; the Year’s website, to which project operators and others were able to upload information regarding their activities and results; and individual activities and projects undertaken throughout the Year, such as the closing conference, nationally organised conferences and projects focusing specifically on the exchange of good practice.

The DAC’s members generally considered that it had contributed positively to an exchange of knowledge and experience among the participating countries. However, it must also be noted that the exchange of good practice in this forum was based on short briefings by the Member States and the Commission, and does not appear to have been based on a formal strategy applying previously defined parameters or criteria.

Evaluation Question No. 4:

Evaluation of the entire implementation process and delivery mechanism: implement the EYPD decentralised approach, mobilisation and participation from the relevant actors (including corporate sector), promotion of new partnerships, empowerment of the persons with disabilities, complementarity of the interventions at national, regional, and local level. Relevance and effectiveness of the preparatory phase in 2002, incidence on the success of the year in 2003. Means put in place at community level to insure the EU information campaign, the dissemination and the transfer of practices and their mainstreaming. Relevance of the choice of delivery mechanisms and possible alternatives

Due to the multi-layered nature of this evaluation question, our summary takes a concise stepwise approach.

Implementation of the EYPD decentralised approach

The decentralised structure of the EYPD was arguably the key aspect of the chosen delivery mechanism, since more than half of the total EC budget was allocated to the responsible national authorities. In practice, the countries differed substantially in the timeliness of their preparatory actions (see further below). Two main factors seem to have influenced the implementation of the Year: (i) The co-operation with the European Commission in relation to the EYPD’s decentralised structure, and (ii) Internal organisational factors.

Firstly, the representatives of the national authorities expressed a high degree of satisfaction with the EYPD’s decentralised structure, and the participating countries responded positively to being in charge of their own preparations for the Year. In the survey among the members of the DAC and the NCBs, it was clearly apparent that the decentralised approach had improved the scope for adapting the EYPD to the national contexts of the individual countries.

According to about one third of the participating countries, the decentralised approach added to the administrative burden of the Year, which, on top of the tasks involved in co-ordinating with the European Commission, was too time-consuming and too onerous in relation to the magnitude of the funding. The associated administrative work related to the management of the global grant and reporting to the Commission, and inter alia included the submission of management reports every 6 months, outlining the activities and their cost in connection with the payments received from the Commission, as well as the contracting requirements connected with the project holders and contractors. In this regard, it is notable that the Commission did not define any standardised monitoring procedures or systems to enable the countries to gather data concerning their activities. The evaluator does not, in the context of the EC regulations, consider the administrative requirements as having been too rigid.

Secondly, delays in the preparatory phase arose for a number of different reasons. It was therefore decided to extend the duration of the Year to the end of March 2004.

Mobilisation and participation of the relevant actors (including the corporate sector)

According to Article 2 of the Council Decision, the objectives of the Year included the reinforcement of co-operation among all the parties concerned. Also, Article 3 of the Council Decision called for co-operation with broadcasting and media organisations. A mapping of the National Co-ordinating Bodies shows that most countries succeeded in mobilising a variety of types of actors. Overall, it is notable that all countries conformed to the Council Decision with regard to representing the disability organisations in their national co-ordinating bodies and involving them in the planning and implementation of the Year.

Promotion of new partnerships

The study showed that the majority of the projects funded at national, regional and local levels were based on partnerships established during prior co-operation.

Empowerment of the persons with disabilities

At all levels of the intervention it appears that the inclusion, involvement, and active participation of people with disabilities and their representatives was a key goal that also succeeded in practice. For example, this is indicated by the project operators’ responses contained in the two tables below.

Table 4-4 Involvement of people with disabilities in the planning of projects

To which degree were people with disabilities and disability organisations involved in the planning of the project? - B) People with disabilities | Respondents | Percent |

To a high extent | 178 | 58,4% |

To some extent | 77 | 25,2% |

To a limited extent | 31 | 10,2% |

Not at all | 15 | 4,9% |

Don’t know | 4 | 1,3% |

Total | 305 | 100,0% |

N = 302

Source: Ramboll Management survey among project operators

Table 4-5 Involvement of people with disabilities in the running of projects

To which degree were people with disabilities and disability organisations involved in the actual running of the project (project management, key project activities, etc.)? - B) People with disabilities | Respondents | Percent |

To a high extent | 166 | 55,0% |

To some extent | 76 | 25,2% |

To a limited extent | 38 | 12,6% |

Not at all | 19 | 6,3% |

Don’t know | 3 | 1,0% |

Total | 302 | 100,0% |

N = 298

Source: Ramboll Management survey among project operators

On another note, it is the evaluator’s assessment that the stipulation that people with disabilities or their organisations must be involved was a key concern for the National Co-ordinating Bodies of all countries. This is evident from the selection criteria contained in the calls for proposals, the kinds of operators chosen, and the operators’ own assessments of the implementation.

Complementarity of the interventions at national, regional and local levels

Some countries, such as the Netherlands and Finland, gave particular priority to one level of intervention, and sought to complement this strategy through their organisational set-up, but in most countries mutually complementary actions were undertaken at all levels. Approximately 70% of projects were carried out at the local and regional level, 18% at the national level, and around 12% were transnational projects.

In several countries, actions at the national level, such as information campaigns, were used for informational purposes to create general awareness of the EYPD and the rights of people with disabilities. These activities thereby assisted many smaller projects at the regional and local level, as they helped to build a general momentum for these projects that could be used as a stepping-stone.

Additionally, national conferences were frequently used to present information concerning general policy developments, and also to exchange information and good practice between key stakeholders and the results obtained by the projects. Additionally, where relevant, national websites and the Community’s EYPD website were often used to create complementarity between the various levels. This was achieved by inter alia allowing projects to upload information about their events and activities, which contributed to their overall visibility.

Relevance and effectiveness of the preparatory phase in 2002 - Incidence on the success of the year in 2003

The Commission stressed the importance of the preparatory activities as a prerequisite for achieving the objectives of the intervention. Evidence from the DAC meeting minutes shows that the Commission was timely in meeting the planned deadlines for the majority of its activities.

Means put in place at community level to ensure the EU information campaign

The Community-wide actions must also be viewed in the context of the decentralised structure of the Year and the relatively large array of projects conceived at national level. Logically, the role of the Community-wide actions, notably of those occurring as part of an information campaign, should be used to unify the disparateness of the projects in order to contribute to the exchange of good practice, networks etc., as prescribed by the objectives of the intervention.

The working relationship between the Commission and the PR agent contracted to ensure the implementation of the campaign was based on standard EC contracts for such services. In practical terms, frequent meetings were held between the Commission, Ogilvy and the EDF in order to plan and implement the campaign. After a comparison of the contracted services envisaged with the actual output produced, the evaluator concluded that adequate human and financial resources had been obtained for the campaign.

The dissemination and transfer of practices and their mainstreaming

As was concluded in relation to evaluation question 3, the exchange of good practice mainly occurred through three types of instruments/forums, namely the DAC; the Year’s website, to which project operators and others were able to upload information regarding their activities and results; and individual activities and projects undertaken throughout the Year, such as the closing conference, nationally organised conferences and projects focusing specifically on the exchange of good practice.

Relevance of the choice of delivery mechanisms and possible alternatives

Overall, the evaluator found that the decentralised structure was the most viable option for the implementation of the Year. This assessment was based on the following argument, which is recapitulated from Section 4-5 in the report. (i) As noted above, the chosen delivery mechanism was pertinent to the principle of subsidiarity by virtue of the allocation of key decision-making responsibility to the national authorities. (ii) We found that the decentralised structure arguably led to a better targeting of the Year in terms of the evolving needs in the participating countries as compared with a more centralised approach. (iii) At a more operational level, the decentralisation also allowed strong targeting of the Year in relation to potential project operators, since in many cases the actors in the NCBs and responsible national authorities had previous experience of co-operating with them and were arguably better situated for reaching many of the smaller actors, which facilitated a greater degree of mobilisation. (iv) We found that the decentralised structure tended to create a greater degree of ownership among relevant actors such as disability organisations.

Evaluation Question No. 5:

Role of the different actors regarding the effectiveness in achieving the objectives, role of the Commission services, Committee support structures, national, regional, and local authorities and other organisations, NGOs. Representatives of people with disabilities, and/or their families, EU structures financed in the field of specific disability groups.

The evidence presented in Section 4.1.2 showed that the Disability Advisory Committee’s implementation of the Community programming and the Commission’s preparatory activities were generally undertaken in line with the planned schedule. This facilitated a timely and effective overall implementation of the Year, and hence its ability to achieve its stated objectives.

With regard to the roles of the various actors, it is well documented that the DG EMPL Unit ‘Integration of People with Disabilities’ sought to use the Commission’s Inter Service Group on Disability as a forum for maximising the attention given to the EYPD in other Community actions. In this way, representatives of other Commission Services and other units within DG EMPL (e.g. DG EAC and the Anti-discrimination Unit) were represented at various DAC meetings or consulted, in order to create the maximum awareness of the Year through other Community interventions.

No less important was the active co-operation with the EDF and, to some extent the EASPD, in the planning and implementation of the Year. In this context, the Commission’s enduring commitment to such multilateral consultations in the 2003 Action Plan is also a significant indication of the importance it attaches to co-operation with these main stakeholders.

Additionally, mention should be made of the European Parliament, which organised the European Parliament of Disabled People, and of the Economic and Social Committee and the Committee of the Regions. Both the European Parliament and the Committees expressed strong support for the EYPD through their resolutions and opinions on the EYPD and the 2003 Action Plan, and via their participation in the events that took place during the Year.

Additionally, the social partners at EU level signed a declaration supporting the employment of people with disabilities as a contribution to the Year.

Most countries succeeded in mobilising a variety of types of actors at the national programming level, ranging from national, regional, and local authorities to social partners, disability organisations and NGOs. Similar involvement, especially of people with disabilities and disability organisations, was evident at project level, and is also documented in the tables above.

A key aspect of the success of the Year was the significant extent of ownership at the different levels of the intervention. The Year was organised as a people’s campaign under the slogan ‘Get on board’, clearly demonstrating that the organisation of the events, the implementation of the projects and hence the attainment of the results relied heavily on the commitment of those involved, both at the programming level and the project level.

The programming level was important not least because of the large degree of decentralisation that characterised the Year, and commitment at the national programming level was therefore essential for the success of the implementation. The fact that this existed is among other things attested to by the large number of stakeholders that were involved, as well as by the extent of their involvement. The stakeholders included people with disabilities, their organisations, the social partners, public authorities and many others. Another indicator of the level of commitment is the financial return on investment at the Member State level, which in most countries greatly surpassed what was envisaged by the Council Decision.

While the decentralised approach required a large degree of commitment from those involved, it was also in itself an important factor for creating the necessary ownership. As has been described elsewhere, decentralisation, the overall delivery mechanism and the breadth of objectives allowed the Member States to adapt the Year to the challenges that each country considered most urgent. The large degree of influence that was given to the Member States and the stakeholders involved clearly contributed to this ownership and commitment.

At the operational level, it is perhaps less surprising that the project operators should have been committed to their individual projects, but the large number of project applications received in most countries clearly attests to the general interest in the issue. What should also be mentioned is the commitment demonstrated by the large number of events that were organised in celebration of the Year and received no official funding.

Evaluation Question No. 6:

Evaluation of the efficiency: cost-benefit analysis of the choice of Community methods of action, in particular the EU communication campaign, the EU events. Repartition of funding in different strands and objectives and the co-financing rate.

As with a great many socio-economic interventions, any judgment of the efficiency of the intervention is hampered by the lack of adequate standardised units of measurement for the outputs and outcomes produced by the intervention.

At the Community level, the efficiency of the Year relates to the output and outcomes of the Year in relation to the human and financial resources that were invested. The unit assigned a full-time project manager to the task. Additionally, a number of other staff from the unit participated in the implementation of the Year on a more limited basis, bringing the total human resources to around 5 FTE.

There are no data that suggest that these human resources were not efficiently deployed. However, while most of the activities for which the Commission was responsible were undertaken as planned, some were delayed, with the primary example being the call for tenders for the evaluation, which was delayed by more than a year due to a lack of human resources. This suggests that the amount of human resources available for the preparatory phase was rather on the low side, although the performance of the activities that were essential for the implementation of the Year per se was not affected.

We refer to Table 2-1 regarding the financial resources. As for the CWA actions that were procured through an open call for proposals (the transnational projects that were launched and co-ordinated at EU level, plus the NGO information campaigns), it is noteworthy that the degree of co-financing from other sources almost exactly matched what was envisaged in the Council Decision (EC: 80%, other sources: 20%). Thus the return on investment did not exceed this level.

It is not possible to assess the extent of all the financial and human resources that were invested in the Year at national, regional and local levels, as much of this investment was undertaken at the local and regional level as a component of activities that were not governed by the NCBs or other involved bodies at the national level.

The analysis of efficiency makes explicit the difference in funding strategies when one examines the average EU and total funding per project (see Table 5-4). The average total funding per project ranges from EUR 2,000 to EUR 50,000.

Table 4-6 Actual total of funded projects in the participant countries[5]

Total number of projects accepted | Total EU co-funding expended on projects | Total co-funding for projects from other sources | Total funding for individual projects | Average EU co-funding per project | Average total funding per project |

Austria | 22 | 299,491 | 300,427 | 599,918 | 13,613 | 13,656 |

Belgium | 197 | 276,532 | 580,855 | 857,387 | 1,404 | 2,949 |

Denmark | 30 | 99,763 | 284,235 | 383,998 | 3,325 | 9,474 |

Finland | 26 | 158,450 | 185,462 | 343,912 | 6,094 | 7,133 |

France | 112 | 480,000 | 5,613,474 | 6,093,474 | 4,286 | 50,120 |

Germany | 151 | 456,621 | 6,817,271 | 7,273,892 | 3,024 | 45,147 |

Greece | 16 | 349,715 | 414,793 | 764,508 | 21,857 | 25,925 |

Iceland | 12 | 26,706 | 24,433 | 51,139 | 2,226 | 2,036 |

Ireland | 45 | 189,333 | 505,662 | 694,995 | 4,207 | 11,237 |

Italy | 22 | 800,000 | 800,005 | 1,600,005 | 36,364 | 36,364 |

Luxembourg | 9 | 50,000 | 36,825 | 86,825 | 5,556 | 4,092 |

Norway | 46 | 128,199 | 128,199 | 256,398 | 2,787 | 2,787 |

Portugal | 34 | 149,954 | 917,676 | 1,067,630 | 4,410 | 26,990 |

Spain | 53 | 360,000 | 717,370 | 1,077,370 | 6,792 | 13,535 |

Sweden | 49 | 235,566 | 225,263 | 460,829 | 4,807 | 4,597 |

The Netherlands | 157 | 671,446 | 972,096 | 1,643,542 | 4,277 | 6,192 |

UK | 170 | 800,000 | 2,616,844 | 3,416,844 | 4,706 | 15,393 |

Total | 1,151 | 5,531,776 | 21,140,889 | 26,672,665 |

Total average | 4,806 | 18,367 |

Source: Ramboll Management

Concerning the project level, we can restate that the Community’s funding of the projects was not merely doubled by national funding (as stipulated in the Council Decision), it returned EUR 4.5 on each EUR 1 invested.

The available evidence indicates that the operators considered that their resources were adequate for attaining their intended results, but that having fewer resources available would have impeded their attainment. About half of the operators stated that the availability of additional resources would have helped them to achieve much better results.

The figure below shows the financial allocation of resources in terms of the different objectives of the Year. The figure shows that almost two thirds of the financial resources were spent on achieving the outcomes of raised awareness concerning disability issues, improved communication regarding disability, and the promotion of a positive image of people with disabilities.

Figure 4-1 Financial trail analysis[6]

Evaluation Question No. 7:

Evaluation of the synergy between activities at national and European level against the objectives of the EYPD.

Synergy between European-level and national-level activities can especially be ascribed to the Information and Promotional Campaign. The Bus succeeded in drawing a considerable amount of media attention. Furthermore, the common logo and the promotional material can generally be said to have underlined the European dimension, and consequently to have encouraged mobilisation, by helping to establish a feeling of belonging to a European-scale intervention.

On the other hand, a number of national authorities expressed doubts concerning the success of the Bus in representing the leitmotif of the intervention as intended. However, overall we find that the evidence supports the conclusion that the Bus achieved a number of good results in terms of synergy, for instance by attracting media attention, drawing a large number of visitors (80,000), and being a venue around which to organise events.

Regarding the transnational projects (i.e. those projects that were launched and co-ordinated at the EU level), the fact that they were not properly synchronised with the Year and the rest of the activities taking place greatly hampered their ability to create synergy and to contribute directly to the Year.

The EU-supported NGO information campaigns aimed to promote information sharing and networking among the disability organisations and individuals with disabilities in order to make them aware of the local, regional, national or Community activities that were occurring during the EYPD. In this way the campaigns were clearly focused on the creation of synergy with other actions. However, the relatively low number of visitors to the websites indicates that the synergy achieved may have been limited.

Table 4-7 Contributions of Community-wide activities

To which extent did the Community-wide activities contribute to the results of the EYPD in your country? |

To a high extent | To some extent | To a limited extent | Not at all | Don’t know/not relevant | Total |

A) The Bus (the March) | Count | 2 | 10 | 8 | 5 | 1 | 26 |

Percent | 7,7% | 38,5% | 30,8% | 19,2% | 3,8% | 100,0% |

B) The Corporate Partnership Programme | Count | 2 | 6 | 8 | 6 | 4 | 26 |

Percent | 7,7% | 23,1% | 30,8% | 23,1% | 15,4% | 100,0% |

C) The EU information and promotional campaign (website, support for local campaigns, information material, logo and slogan) | Count | 6 | 12 | 6 | 1 | 1 | 26 |

Percent | 23,1% | 46,2% | 23,1% | 3,8% | 3,8% | 100,0% |

N = 26

Source: Ramboll Management survey among members of DAC, NCBs, and HLG

Evaluation Question No. 8:

Evaluation of the complementarity with other Community instruments and programmes such as programmes to combat Anti-discrimination, social exclusion, gender equality, EQUAL initiative and Youth programme

Based on the examination of related Community interventions, the evaluator found that several of the objectives of the EYPD cohered well with the objectives of the anti-discrimination programme, the programme to combat social exclusion, the EQUAL initiative’s Youth programme and the Socrates and Leonardo da Vinci programmes, and were therefore complementary to these.

At an operational level, it can be noted that it was possible to use some programmes to support specific actions linked directly to the EYPD. Among others, projects falling under theme 1 of the so-called ‘Joint actions’ carried out as part of the Socrates, Leonardo da Vinci and Youth1 programmes focus on the integration of people with disabilities, and relate directly to the EYPD.

Also, a number of concrete activities were undertaken in relation to EQUAL which supported the aims of the EYPD, including the publication of an EQUAL good practice report in the field of disability. The European Thematic Group also had a special focus on disability during the Year.

Finally, the Community Action Programme to combat discrimination was used to fund the European Day of Disabled People 2002 that among other things was used to gather support and interest for the EYPD.

Through such complementary activities it was effectively possible to increase the budget that was allocated for the objectives of the EYPD.

At an operational level, the Commission’s Inter Service Group on Disability was a key forum for maximising the attention given to EYPD in other Community actions.

Arguably, a synergy existed between the efforts pursued through the above-mentioned programmes and the EYPD, and the unique nature of the Year clearly justified any overlaps that occurred between them. The impact of the other programmes in the area of disability and non-discrimination created the foundation for a successful EYPD. This was further ensured by the Commission’s Inter Service Group on Disability which co-ordinated the efforts of the other programmes and those of the Year. It was thereby actively able to increase the emphasis given to the area of disability during the course of the EYPD and subsequently disseminate the results of the Year so that they could potentially be built on by the other programmes.

Evaluation Question No. 9:

Evaluation of EYPD’s impact reached compared with the needs of the target populations. To what extent can the positive changes be expected to last after the EYPD has been terminated? What is the potential impact on the targeted populations?

Community level;

As has been described, a main outcome was clearly the adoption by the Commission of the communication “Equal opportunities for people with disabilities: A European Action Plan”.[7] The Action Plan builds on the Commission’s strategic goals in the disability field, which are to be achieved by three operational objectives:

- Achieving full application of the Equal Treatment in Employment and Occupation Directive and launching the debate concerning the future strategy to combat discrimination

- Successful mainstreaming of disability issues in relevant Community policies and existing processes

- Improving “Accessibility for all”

In the evaluator’s assessment, the EYPD therefore appears to have influenced the preconditions for future action in the disability field positively, and has contributed to the further integration of disability considerations in other policy fields. However, the question remains as to whether the momentum created by the EYPD can be maintained in the long term and will actually generate lasting impacts. While it is still too early to draw firm conclusions, the adoption of the Action Plan may help to ensure just that. Accordingly, the Action Plan and the accompanying list of action items identify the concrete steps to be taken and the various DGs which are to be involved in helping to maintain focus and momentum. The Action Plan may accordingly contribute to the creation of conditions that will promote the employment of people with disabilities.

While the specific initiatives in the Action Plan focus on employment, an additional aim of the Action Plan is to mainstream disability issues into all relevant EU policy areas and at all stages of policy development. The Commission has made a commitment to this aim, which is further supported by the work of the Commission’s Inter Service Disability Group (ISDG). Increased mainstreaming is one of the initiatives that could clearly have a more lasting impact.

In addition to the initiatives launched by the Commission during the EYPD, the political commitments made by the European Parliament and the Council through the adoption of resolutions and conclusions show that there is a potential for building on the momentum created by the Year at European level. In themselves these commitments do not, however, lead to any lasting change, and must be followed by concrete initiatives and action if actual impacts are to occur. It is worth noting that future action in this sense is not only relevant to the European Parliament and the Council. As part of their commitment, the institutions asked the Commission and the Member States to mainstream disability issues into their policies wherever possible, and to reinforce their co-operation on disability policies at the EU level.

National level

At Member State level, the possibility that the EYPD might to lead to lasting changes stems from the outcomes produced by the individual actions, and especially from the possibility that the EYPD could also function as a lever for further action.

As has been previously concluded, in many cases the Year enabled organisations and stakeholders to gain access to decision-makers and the decision-making process. Such contacts established during the Year could also contribute to a greater degree of involvement of people with disabilities and their organisations in the decision-making process in the longer term. In this regard the Year may also have improved the ability of people with disabilities to influence their own situation.

The evidence presented suggested that in several cases the Year prompted decision-makers to act on disability issues, and that they used the Year as a platform to launch initiatives. A large number of new legal and policy initiatives were introduced during the Year. The available data does not permit clear conclusions as to why policy and legislative changes took place in some countries and not in others. However, the following suggestions can be offered.

Firstly, the baseline situation in terms of legal protection, access, equal opportunities etc. is an important issue in any consideration of the presumed requirement for legal changes.

Secondly, different strategies were adopted in the different Member States. In some countries such as Spain, a clear intention of using the Year as a lever for new initiatives existed, whereas in Finland this issue was not given priority. Conversely, Finnish disability policy underwent significant revision in 1995, and the need for new legislation was therefore not subsequently considered to be as acute.

Thirdly, the organisational strength and influence of disability organisations and their ability to use the Year as a lever varied.

Fourthly, in some cases the EYPD may have been ‘out of synch’ with the ongoing political process, such as in France, where revision of the principal legislation in the disability field has been underway since 2000-2001.

At the project level, the evaluation documented potential sustainable impacts in the form of networks among those disability organisations that engaged in partnerships in the course of their projects. Those surveyed indicated that future collaboration was quite likely, which may contribute to future results.

We consider this potential as something that must be nourished in order to be sustained. Accordingly, sustainability at the project level may therefore be dependent on renewed opportunities for interaction and project collaboration. The failure to create incentives for (or to foster) such collaboration may result in the networks fading away. As with much other civil society organisational work, the availability of resources is a key issue in this context.

Conclusions and recommendations

This chapter contains the key conclusions of the evaluation and the corresponding recommendations derived from these conclusions. The recommendations are broken down into the following steps:

- Overall recommendation

- Operational action(s)

- Actors involved

- Feasibility

- Timing

The evaluation finds that an information campaign was an appropriate tool in addressing the challenges facing people with disabilities at a European level. Similarly, the objectives, whilst broad, can be said to address these challenges. At a national level a large degree of pertinence between the objectives of the EYPD including the national priorities and the evolving needs in the participating countries.

The evaluation shows that the decentralised structure of the delivery mechanism coheres with and supports the principal of subsidiarity and thus the overall decision-making structures in the area of social policy which must be said to be the point of departure for many disability issues. Whilst implying an inherent risk that the focus on the objectives, coherence, and coordination will be lost, the structure also support the relevant decision-making structure in the field and thus supports an optimal prioritisation of resources compared to the problems identified at national level.

Recommendation No. 1:

Overall recommendation:

The evaluator recommends that the decentralisation instrument should be carefully considered in policy interventions where the decision-making competence is located at the level of national government, in accordance with the subsidiarity principle.

Operational action:

Since the specific objectives in the EYPD (or for that matter in a number of other Community interventions) are not very concrete, the evaluator proposes that the utmost consideration should be given to requiring each national authority to define its own national priorities. The priorities should be defined in the form of the operational objectives and concrete results that are envisaged at national level.

Principal actors involved:

European Commission, national authorities.

Feasibility:

Given the evaluation results for the EYPD, the national authorities have emphasised the importance of its pertinence to decision-making competences, its scope for allowing concrete problems to be addressed in the national context, and its creation of intrinsic ownership of the intervention at the national level. We therefore anticipate that this delivery mechanism can be transplanted into other policy contexts.

Timing :

This consideration must be included in the ex-ante evaluation of the Commission Services.

The evaluation concluded that, given the available evidence, that the planned output and results were in fact realised.

Community level

Based on the available evidence it may be concluded that the Year has worked as a lever for the disability issue at Community level in the sense that it has been agenda defining and also has prompted decision makers to act upon disability issues and especially in terms of the Council to take a stand on the disability issue.

In the evaluator’s assessment, the EYPD thus appears to have influenced the preconditions for future action in the disability field positively and contributed to further integration of disability considerations in other policy field.

Reception of supplementary grants

The evaluator does not find the procedure of the European Parliament awarded a supplementary budget specified for further Community Wide Activities late in the process as strategically appropriate. We estimate such procedures are relatively time consuming, and with limited alignment to the overall strategy. Thus, limited human resources are spent on procuring and monitoring budgets instead of maximizing the exchange with the national level.

Recommendation No. 2: Overall recommendation: For future interventions, we propose that activities are synchronised during a specified period of the intervention. Operational action: The programming document should clearly identify the milestone in the preparatory phase, i.e. the deadline for supplementary budgets. For instance, this might occur two months prior to the planned deadline for a call for proposals for Community-wide actions. Principal actors involved: European Commission, European Parliament Feasibility: The procedure must be as agreed with the European Parliament, but the administrative burden and synchronisation problem must be taken into account when the procedure is agreed. Timing: Preparatory phase prior to intervention. No less than two months prior to the call for proposals for Community-Wide Actions. |

Good practice

Notwithstanding, that exchange of good practice was possible and did take place through several forums and at different levels it may also be concluded that the exchange of good practice did not follow from any concerted or systematic strategy that has ensured a systematic collection or dissemination of good practice across the participating countries, but mainly occurred more randomly at select activities.

Recommendation No. 3: Overall recommendation: The evaluator recommends a more concerted and systematic strategy for the exchange and dissemination of good practice. This means that the themes and criteria for good practice must be further refined. This can be done at the Community level. The participant countries would also report their good practice cases at this level. Again, cross-cutting good practice criteria are needed at the project level. For example, a project that has attained (self) reported excellence could be introduced to transnational networks in order to facilitate such an exchange among the project operators. The data generated by these operators would subsequently be disseminated. Operational action: Cross-cutting criteria for good practice can be set out in multiple documents. Since they will cover a wide variety of actions, the criteria must be fairly generic. For instance, we propose the following criteria for an information campaign: Publicity (Evidence of reaching the target population) Transparency (Actions can be replicated and transferred to other contexts) Professionalism (The operator’s performance represents the state of the art in the field) Performance (Evidence of achievement of expected outcome) Innovation (Evidence of having produced innovative products or outcomes) Sustainability (Evidence of the likely mainstreaming of the outcomes in the field) Principal actors involved: European Commission, national authorities, project operators Feasibility: The definition of the legal basis should be contained in a Council Decision or in the Advisory Committee’s rules of procedure. The Commission must ensure financing for good practice dissemination. The identification of good criteria must be mutually agreed with the authorities in the participant countries. This can be done via Council Decision. Technical dissemination (e.g. websites, booklets, conferences) must be feasible using the information campaign mechanism. Timing: Ideally via Council Decision. At the latest, identification of good practice criteria must be done prior to the commencement of actions at the European and national level. Ideally, dissemination should take place during the intervention. However, it is more likely to take place following the termination of the intervention. |

- National level

Among the arguments for the EYPD was its predicted ability to act as lever or catalyst for further actions and results. The data suggests that amongst others that the Year and the media exposure created by it enabled various actors to define themes for the public debate on disability issues that would not otherwise have been conveyed and helped different organisations and stakeholders gain access to decision makers and the decision-making process.

Ex ante evaluation

The evaluator notes that the ex-ante evaluation was of a fairly general character, and did not establish a solid baseline and indicators for the attainment of the anticipated results of the intervention.

Recommendation No. 4: Overall recommendation: Further emphasis on the ex-ante evaluation at Community and national level Operational action: The Commission’s Activity Based Management regulations state that further emphasis should be placed on the ex-ante evaluations[8]. The use of such evaluations is to be mainstreamed within the Commission services. For similar Community interventions, we recommend that the ex-ante evaluation instrument should be further refined. In line with administrative regulations, we recommend that this should subsequently be regarded as a key policy document for attaining more tangible expected results. As part of the ex-ante evaluation, an analysis of the intervention logic will probably reveal the need to establish a baseline situation in order to be able to assess foreseen outcome and impact. We recommend that the planned development of disability context indicators should be related to assessments such as the key, generic, and specific indicators for the interventions[9]. Principal actors involved: European Commission. Feasibility: According to ABM regulations, ex-ante evaluations are to be mainstreamed in the Commission Services as of 2004. Therefore, if the Commission meets its own guidelines this will be implemented in the future. Timing: Prior to the Council Decision drafting proposal. |

The Commission only requested basic financial data from the national authorities concerning this EC-funded project. On the admission of the Commission’s administrators, this procedure was agreed in order to reduce the administrative burden on the national authorities. However, this was at the cost of establishing more solid data concerning the expected outcomes, and thus of achieving a better basis for assessing the effectiveness and efficiency of the intervention. This could have been accomplished if the Commission had ensured that a common monitoring and information system was in place for all the countries involved.

Recommendation No. 5: Overall recommendation: The Commission should devise a standardised monitoring system that would enable the countries to accumulate data concerning their activities. Operational action: The Commission should devise a monitoring system listing basic information about the planned activities and expected results. We refer to the Annex concerning the details of a proposed system. It is enough to say here that the basic parameters should be: Contact details of operator Kind of operator Name and ID code of project Kind of objective and Action being pursued Kind of planned output Kind of expected result Total budget Contributions from EC, national and private sources Principal actors involved: European Commission, national authorities, external evaluator. Feasibility: Basic concepts should be identified in the ex-ante evaluation and may be subject to further elaboration in the later stages of the preparatory activities. Timing: The design of the system should be undertaken in the middle part of the preparatory phase. The data entry should take place no later than one month after the selection of the projects. |

- The evaluation documented that the majority of preparatory activities carried out by the Commission was done in time. The tender procedure for the final evaluation suffered delay. The evaluator supports the Commission’s emphasis on the importance and timeliness of the preparatory activities. The timeliness of the preparatory activities by the participant countries varied. Most countries did manage to establish key structures and execute key actions in time. Some countries were late. No direct connection to the reported results can be drawn, however. However, exchange of good practice and synergy in a multinational intervention is clearly dependent on timing. The delay in preparation may not necessarily result in documented lesser effect at national level, but may impede exchange and synergy at European level.

Recommendation No. 6: Overall recommendation: The Commission may consider enforcing the timeliness of preparatory activities by imposing deadlines in the legal framework. Operational action: The exchange of good practice and synergy in a multinational intervention is clearly dependent on timing. As has been demonstrated, a number of countries were late in implementing their preparatory activities. The delay in preparation may not necessarily result in a demonstrably lesser effect at the national level, but may impede exchange and synergy at the European level. Therefore the evaluator proposes that deadlines for the preparatory activities should be included in the legal framework governing the awarding of global grants. This may ensure that the opportunity for synergy will not be wasted due to a lack of synchronisation. The availability of ample preparation time is a key issue for a successful intervention. Given the human resources invested at both the European and the national level, we consider 18 months from the adoption of the Council Decision to the commencement of the Year to be an adequate timeframe. A proposal for milestones at the national and Community levels might be as follows: Month 1 / Establishment of national co-ordinating body Month 2 / Development of draft strategy Month 3 / Establishing of contact with Inter-Service Group Month 6 / Definition of national priorities Month 7 / Establishment of standard monitoring system Month 7 / Deadline for supplementary budget Month 8 / Start of preparations for launch events Month 9 / Launch of Call for proposals Month 10 / Evaluation of proposals, external evaluation of Call for proposals Month 11 / Notification of results of Call for proposals Month 12 / Submission of data to be entered into monitoring system Month 14 / Launch of national website, national information days Month 16 / National information days Month 18 / Launch event We further propose that certain milestones (e.g. the establishment of national priorities, Call for proposals etc.) must be reached as a prerequisite for signature of the global grant agreement. Principal actors involved: European Commission, national authorities. Feasibility: It is likely that some national authorities will be late regardless of the time available for preparation, and therefore we consider that the combination of providing financial incentives to meet milestones with the granting of adequate time may be a useful instrument. Timing: As stipulated above. |

- All countries were found to adhere to Council Decision’s specification of mobilizing relevant actors. Likewise, the decentralised structure enabled the national authorities and disability organisations to devise strategy and design the implementation to exploit complementarity at national, regional, and local level.

The evaluator finds that the decentralised delivery mechanism has be a strong vehicle and significant factor in the success of the intervention.

Recommendation No. 7: Overall recommendation: Implementation of the decentralised structure. Operational action: The evaluator recommends that the national priorities put in place by the national authorities should be defined as operational objectives in the legal framework. Similarly, the national authorities should be required to document their planned results and to submit indicators that quantify the baseline situation as well as the results. Principal actors involved: European Commission, national authorities. Feasibility: The Council Decision should proclaim the status of the national priorities as operational objectives. Likewise, the Decision or the global grant agreement must specify the monitoring and evaluation responsibilities of the national authorities. Taking into account the consent given to such an arrangement in EYPD 2003, we consider this action to be feasible. Timing: Submission of national priorities should take place no later than six months after the commencement of the preparatory phase. |

The evaluator documented that most countries have succeeded in mobilising a variety of types of actors ranging from relevant national, regional, and local authorities, social partners, disability organisations, and other NGOs at national level. A similar involvement of especially, disability organisations was evident at project level. The evaluator finds the regulation in the Council Decision were thus an effective vehicle in mobilising and enabling the active participation of people with disabilities and their organisations.

Recommendation No. 8: Overall recommendation: The evaluator recommends the Commission to encourage multilateral collaboration in certain contexts. Such collaboration has already been anticipated by the Commission in the new European Action Plan. Operational action: We propose that multilateral consultation should take place with regard to the EU action plan. In particular, we consider that consultation is crucially important when designing interventions in e.g. employment (in the Action Plan’s first phase). The consultation may involve commenting on the prioritised actions and instruments. Likewise, such involvement can be built into the legal basis of the interventions, similar to the case of the EYPD – for instance, concerning special actions/strands in EQUAL. Principal actors involved: European Commission, EDF, EASPD. Feasibility: The extent of the consultation and involvement occurring may well depend on the political will of the Commission. Given the intentions expressed in the Action Plan, we consider it to be feasible in the medium term. Timing: Discussions should take place in the Commission’s Inter-Service Group on Disability immediately. |

The results of the evaluation point toward the benefits of integrating the disability organisations in the Commission’s planning and implementation of like interventions. This has also been documented through the results at national level. Such structures of cooperation were already well established in a number of countries prior to EYPD. Nevertheless, the available evidence suggests that these structures at national, regional, and local level have been strengthened through the cooperation.

Recommendation No. 9: Overall recommendation: The evaluator recommends that interventions with similar intentions of strengthening the network of co-operation will include similar stipulations in the legal framework of the Council Decision for similar programmes at national, regional, and local levels. Operational action: Consultation is already well established as an inherent part of the policy formulation process in a number of countries. Since the functioning of the political system is a national responsibility, the Commission may consider using the open method of co-ordination as a means of demonstrating the positive results of extensive consultation. Principal actors involved: National authorities, national disability organisations, association of service providers. Feasibility: It not likely that the Commission can directly influence a major change in the policy formulation procedures. However, the provision of information through good practice examples may be considered appealing in the longer term. We therefore consider the open method of co-ordination to be a feasible instrument in the long term. This could be reinforced by the discussions taking place in the High Level Group on Disability. Timing: This aim should be included in the 2006-2007 work plan. |

While no data suggests that human resources at the Community level were not deployed efficiently the delaying of some activities in the preparatory phase suggests that the amount of human resources available for the preparatory phase was a bit on the low side. It should be underlined, however, that the implementation of activities that were essential for the implementation of the Year per se was not effected.

Recommendation No. 10: Overall recommendation: More human resources in the preparatory phases. Operational action: A central challenge for the Commission in the preparatory phase was the human resources allocated to the intervention. While this did not result in major delays, most of the substantial work was done by just a few individuals. This made the action vulnerable to the transfer/resignation of key personnel during this crucial phase. Therefore we propose that the key responsibilities should be more equally divided among the 5 FTE personnel allocated to the activity. Principal actors involved: European Commission. Feasibility: Insufficiency of human resources in the Commission appears to be a recurrent challenge. Nevertheless, the Head of Unit has the managerial challenge of dividing up the tasks and responsibilities in a more balanced way. Timing: Throughout the preparatory phase. |

The evaluator concluded that, by nature of the intervention, its output and results, reliable cost benefit analyses was not possible as no standard unit of measurement can be defined. However, evidence from the survey with project operator suggest that the actions were carried out efficiently. No other evidence gathered contradicts this proposition.

As reported above, a more detailed monitoring system could have documented the relation between planned and realised output and results.

Recommendation No. 11: Overall recommendation: Similar to evaluation question 4, we recommend the establishment of a standardised monitoring system that will improve the documentation of planned inputs, output and results compared to their actual realisation. Operational action: As indicated above, we consider the existence of a monitoring system to be a key factor in determining the cost efficiency and effectiveness of certain actions and providing an informative basis for assessing what kinds of operators (large/small, experienced/inexperienced, local/regional etc) tend to deliver the best results from the resources invested. Principal actors involved: European Commission, national authorities, external evaluator. Feasibility: Basic concepts should be identified in the ex-ante evaluation and may be subject to further elaboration in the later stages of the preparatory activities. Timing: Design should occur in the middle part of the preparatory phase. The data entry should take place no later than one month after the selection of projects. |

Synergy between European level and national level activities can in particular be ascribed to the Information and Promotional Campaign. Although a number of national authorities expressed doubts concerning the success of the Bus in representing the red thread of the intervention as intended, we do find that the Bus succeeded in drawing a considerable amount of media attention, drawing a large number of visitors (80,000), and being a venue around which to organise events. Furthermore, the common logo and the promotional material in general can be said to have underlined the European dimension and consequently furthered mobilisation by helping establish the feeling of being part of a European scale intervention. This is also reflected in the assessments by the national authorities.

As concerns the transnational projects the fact that they were not synchronised with the Year and the remaining activities taking place hampered their ability to create synergy and contribute directly to the Year.

The NGO information campaigns aimed at promoting information sharing and networking among the disability and individuals with disabilities in order to make them aware of the local, regional, national or Community activities happening during the EYPD. In that way the campaigns were clearly targeted at the creation of synergy with other action. The relatively limited number of visitors on the websites indicates, however, that the synergy obtained may be limited.

Recommendation No. 12: Overall recommendation: To enhance synergy, the Commission must place greater emphasis on ensuring that the organisation and priorities are well defined in the preparatory phase Operational action: Synergy is at least to some extent based on the synchronicity of the actions taking place at both the Community and national levels. The progression of the EYPD showed that a number of countries were late with their implementation, especially with regard to the EC co-funded projects. As above, we propose that further emphasis should be placed on identifying opportunities for synergy by establishing the national priorities and documenting the funded projects in the DAC forum well in advance of the commencement of the Year. The mutual exchange of national priorities and funded projects is likely to enable the exchange of practice, the co-ordination of activities and the creation of optimal leverage effects. Therefore we propose that the establishment of national priorities must be fixed as a milestone to be reached within six months of the preparatory phase. Principal actors involved: European Commission, national authorities. Feasibility: If the Commission is prepared to invest human resources in a thorough analysis of the planned priorities and planned projects and to function as an intermediary in the DAC, further synergy should be possible. Timing: Within six months of commencement of the preparatory phase (national priorities) Within twelve months of commencement of the preparatory phase (planned projects). |

Based on the examination of related Community interventions, the evaluator found that several of the objectives of the EYPD cohered well with the objectives of the anti-discrimination programme, the programme to combat social exclusion, the EQUAL initiative’s Youth programme and the Socrates and Leonardo da Vinci programmes.

At an operational level, it can be noted that it has been possible to use some programmes to support specific actions linked directly to the EYPD, including the Socrates, Leonardo da Vinci, and the Youth1 programmes. Also, a number of concrete activities were undertaken in relation to EQUAL, which supported the aims of the EYPD.

Finally, the Community Action Programme to combat discrimination was used to fund the European Day of Disabled People 2002 that inter alia was used to gather support and interest for the EYPD.

Recommendation No. 13: Overall recommendation: The identification of complementary actions should take place during the preparatory phase. Operational action: The EYPD is a good example of the complementarity in objectives and concrete actions cross-cutting the Commission Services. We recommend that such actions should be emulated in similar interventions. We propose that the ex-ante evaluation should identify what kinds of complementary actions can be envisaged on the basis of an analysis of the objectives and actions/strands of related interventions. Likewise, the Commission Inter-Service Group on Disability may serve as a good forum for exploring and developing relevant interrelationships in order to optimise the effects of the intervention. Principal actors involved: European Commission, Inter-Service Group on Disability. Feasibility: According to the ABM regulations, ex-ante evaluations are to be mainstreamed throughout the Commission’s services during 2004. The Inter-Service Group on Disability is already established. These instruments and structures make the recommendation feasible. Timing: The complementarity assessment must take place during the ex-ante evaluation prior to the proposal for Council Decision. The Inter-Service Group co-ordination should take place in the first part of the preparatory phase. |

Impact at Community level

The overall conclusion of the evaluation is that EYPD helped to strengthen the commitment at Community level by creating a favourable environment for achieving further progress and clearly worked as a lever for the disability issue also at European level.

As detailed a main outcome that is likely to create most impacts was the adoption by the Commission of the communication “Equal opportunities for people with disabilities: A European Action Plan”. In addition to the initiatives launched by the Commission during the EYPD, the political commitments made by the European Parliament (Economic and Social Committee and Committee of Regions) and the Council through the adoption of resolutions and conclusions show that there is a potential for building on the momentum created by the Year and furthering the rights-based approach. It is worth noting that future action in this sense is not only related to the European Parliament and the Council. As part of their commitments the institutions asked the Commission and the Member States to mainstream disability issues in relevant policies and to reinforce cooperation at EU level on disability policies.

Impact at national level

The evidence presented suggested that in several cases the Year prompted decision-makers to act upon disability issues and that the Year was used as a platform by decision-makers to launch initiatives. A large number of new legal and policy initiatives were initiated during the Year. The available data does not permit clear conclusions as to why policy and legislative changes have taken place in some countries and not in others.

At the project level, the examination documents potential sustainable impacts in the guise of networks between disability organisations that engaged in partnerships concerning their projects. Those surveyed, indicate future collaboration is quite likely, which may contribute to future results.

Recommendation No. 14: Overall recommendation: Impacts must be sustained by actions focusing on mainstreaming at Community and national level. Operational action: While the EC action plan marks a significant commitment towards mainstreaming disability issues into all policy fields, a significant challenge exists in establishing a similar commitment at the national level. A significant stepping stone is the Council resolution 2003/C 175/01. Also, the establishment of a concrete work plan for the High Level Group on Disability and the use of the open method of co-ordination are positive steps in the right direction. The evaluator proposes that the Commission should take advantage of the leverage provided by the Year. This should be done by inviting the Member States to present those policy and legal measures which are a follow-up of the resolution adopted by the Council. The forum could consist of a conference or a publication in which the Member States’ actions are presented and discussed. Principal actors involved: European Commission, European Council, national authorities. Feasibility: The feasibility of the actions to be taken at the national level will clearly depend on the national policy context. Nevertheless, we anticipate that the normal pattern of documenting or launching the policy or legal initiatives prompted by the EYPD is feasible for many of the countries. Timing: No later than March 2005 (one year after the completion of the EYPD). |

[1] Spontaneous grants: SP/2002/E04 and SP/2004/E04.

[2] Call for proposals VP/2002/12.

[3] Open call for proposals for transnational projects, VP/2003/14.

[4] Not all Belgian and Spanish projects have been categorised according to the type of expected output. This implies that these figures must be treated with some caution.

[5] The figures for France, Germany, Greece, Iceland, Ireland, Norway, and the Netherlands are based on our information system. The remaining figures derive from the financial information submitted to the Commission by the participant countries.

[6] The financial figures should be regarded as a rough estimate. For Objectives B and D, the distribution is based on an assessment of the type of output produced and on what intermediary level objective it was most likely to produce in the long term.

[7] COM(2003) 650 final

[8] See European Commission Communication “Implementing activity based management in the Commission” (SEC(2001)1197/6 & 7). See also: European Commission: EVALUATING EU ACTIVITIES. A PRACTICAL GUIDE FOR THE COMMISSION SERVICES, July 2004, European Commission, DG Budget (2001): Ex-ante Evaluation. A Practical Guide for Preparing Proposals for Expenditure Programmes, European Commission: Indicators for Monitoring and Evaluation: An Indicative Methodology (The New Programming Period 2000-2006)

[9] As announced in the EU Action Plan. The context indicators will be based on existing (or planned) surveys such as the European Community Household Panel Survey, Labour Force Survey, and SILC.

11