EUROPEAN COMMISSION Brussels, 19.12.2022 SWD(2022) 404 final COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Final evaluation of the Active and Assisted Living Research and Development Programme {COM(2022) 708 final}



Table of contents 1.Introduction 1.1 Purpose and scope of the evaluation 2.What was the expected outcome of the intervention? 2.1 Description of the intervention and its objectives 2.2 Point(s) of comparison 3.How has the situation evolved over the evaluation period? 3.1 Current state of play 3.2 Key Programme statistics 4.Evaluation findings (analytical part) 4.1 To what extent was the intervention successful and why? 4.1.1 Effectiveness 4.1.2 Efficiency 4.1.3 Coherence 4.1.4 Transparency and openness 4.2 How did the EU intervention make a difference? 4.3 Is the intervention still relevant? 5.What are the conclusions and lessons learned? 5.1 Conclusions

5.2 Lessons learned Annex I. Procedural Information Annex II. Methodology and Analytical models used Annex III. Evaluation matrix and, where relevant, Details on answers to the evaluation questions (by criterion) Annex IV. Overview of benefits and costs Annex V. Stakeholders consultations Annex VI. Set-up of the Programme and Additional Programme Statistics



Glossary Term or acronym Meaning or definition AAL1 Ambient Assisted Living Joint Programme AAL2 Active and Assisted Living Research and Development Programme AALA Ambient Assisted Living Association AISBL Association Internationale Sans But Lucratif AHA Active Healthy Ageing AWP Annual Work Programme CMU Central Management Unit DG Directorate-General EB Executive Board EC European Commission EDCTP2 European and Developing Countries Clinical Trials Partnership under Horizon 2020 EIP AHA European Innovation Partnership on Active and Healthy Ageing EU European Union GA General Assembly ICT Information and Communications Technology ISSG Inter-Service Steering Group JPI MYBL Joint Programming Initiative ‘More Years Better Lives’ KPIs Key Performance Indicators MTR Mid-Term Review NCP National Contact Point OO Operational Objective PRIMA Partnership for Research and Innovation in the Mediterranean Area RIE Recommendation of the Interim Evaluation R&D Research and Development R&I Research and Innovation SCPs Small Collaborative Projects SMEs Small and Medium-sized Enterprises TFEU Treaty on the Functioning of the European Union UN United Nations UK United Kingdom 3.1. Introduction 1.11.1 Purpose and scope of the evaluation The purpose of this evaluation is to assess the ‘Active and Assisted Living Research and Development Programme’ (hereinafter ‘the AAL2 Programme’), under Horizon 2020. DecisionNo 554/2014/EU of the European Parliament and the Council on the Union’s participation in the Programme, sets out a pre-established framework for the interim and final evaluation, in line with other initiatives under Horizon 2020 based on Articles 185 and 187 of the Treaty on the Functioning of the European Union (TFEU). The AAL2 Programme is based on Article 185 of the TFEU, which foresees that the Union may make provision for the participation of the European Union (EU) in research and development (R&D) programmes undertaken by several Member States, including participation in the structures created for the execution of those programmes. This final evaluation covers the seven-years of operation of the Programme, from June 2014 to June 2021 and is based on the five evaluation criteria of the Better Regulation Guidelines complemented with two additional partnership-specific criteria of additionality and ‘transparency & openness’ of the Partnership. The final evaluation covers the AAL Association’s Participating States. The full list of AAL2 Participating States for the period 2014-21 can be found in Annex VI, Table 4. Following the impact assessment conducted in 2013 and the Interim Evaluation of the AAL2 Programme carried out in 2017, the final evaluation assesses progress made towards the Programme's objectives and on fulfilling the recommendations set out in the Interim Evaluation. In addition it develops both concrete conclusions and lessons learned covering policy aspects, strategy and operational levels. The findings of this final evaluation draw largely on a supporting evaluation study (hereinafter ‘the study’), conducted by external contractors, which is part of the Research and Innovation Framework Programme Impact Monitoring, Analysis and Evaluation strategy for the period 2019-2024 covering the overall ex-post evaluation of Horizon 2020 and the interim evaluation of Horizon Europe, including partnerships. More particular the AAL2 Programme final evaluation was covered under the evaluation study entitled “Evaluation study on Resilient Europe”. The evaluation is based on the findings of the study, incorporating 2 case studies, using data collected through surveys and approximately 30 interviews with Programme participants and a wider variety of stakeholders, complemented with statistical information about the AAL2 Programme provided by the dedicated implementation structure, managing the Programme. More information on the methodology and process to carry out this evaluation is available in Annex II.

The present evaluation Staff Working Document accompanies a Commission report on the final evaluation of the AAL2 Programme and will feed into the wider debate on how to maximise the benefits of Information and Communications Technology (ICT)/digital solutions for active and healthy ageing through research and deployment coordination, and cooperation with public health and digital health initiatives. 3.2. What was the expected outcome of the intervention? 2.1 Description of the intervention and its objectives The combined challenge of demographic change and digital transformation By 2070, over 30% of Europeans are projected to be 65 years of age or over, compared to just over 20% in 2019. The share of people aged 80 years and older is expected to double from 5.8% to 13.2% over the same period. Within the EU, the impact of ageing varies widely between regions, as well as between urban and rural areas, and gains in healthy life expectancy are unevenly distributed . At the same time there are social and economic opportunities associated with the concept of healthy ageing. Older citizens are increasingly shaping economies and the demand for services and products in the ‘silver economy’ is increasing rapidly . There is now a substantial number of technologies and digital solutions to support people at any stage in their lives, to support ageing in place, enhance the home environment and support social connectedness . In using these technologies to their full potential, there is potential for economic growth and social development in demographic change that can balance some of its impacts on health and (social) care systems and improve the quality of life of the ageing population and their carers. Demographic ageing is set to bring about an imminent and significant change in society and the economy for which the EU is not yet well prepared. Spiralling ageing-related costs for governments are being compounded by missed or underexploited opportunities on the growing markets for ICT-enabled products and services for active and healthy ageing. There are three main problems with these markets. The first is low availability of innovative ICT products and services, caused by EU market fragmentation. Secondly, there is fragmentation of R&D and innovation at European level, with high participation barriers for Small and Medium-sized Enterprises (SMEs). The third problem stems from limited adoption of innovation, related to a lack of evidence building and sharing. The EU added value is evident given these problems: fragmentation of the market and of research efforts, a lack of focus on trans-EU deployment and a lack of a shared European vision on the markets for ICT/digital products and services to assist active and healthy ageing and living. It is in this context that the Commission provides its support through funding and actions that promote policy cooperation, research and exchange of good practice but also stimulates cooperation structures such as public-public partnerships which aim to develop closer synergies, increase coordination and avoid unnecessary duplication with Union, international, national and regional research programmes. Programme Strategy, objectives and intervention logic The AAL2 Programme, focussing on digital innovation in active and healthy ageing, was initiated with the overarching goal to help address the ageing challenge and turn it into an opportunity for Europe. Following strategic programme objectives were agreed among the founding countries: ·Enhance the availability of ICT-based products and services for active and healthy ageing, to improve the quality of life of older people and their carers, and help increase the sustainability of care systems. ·Create/maintain a critical mass of trans-European applied research, development and innovation for ICT-based products and services for ageing well, in particular involving SMEs and users. ·Leverage/maintain private investments and improve conditions for industrial exploitation by providing a coherent framework for developing European approaches and solutions including common minimum standards that meet varying national and regional social preferences and regulatory aspects. The 2013 impact assessment set-out the following operational objectives [OO] to meet the above AAL2 strategic programme objectives: ·Further improve operational excellence and accountability for the Programme [OO1]; ·Reduce time to market, by facilitating user and industry-driven research [OO2]; ·Facilitate participation for all actors in the innovation chain, in particular SME, end-users and service providers, from the start and in all stages of the projects (e.g. through iterative and design and development approaches) [OO3]; ·To increase the number of Participating States and to leverage private and national co-financing [OO4]; ·To ensure complementarity with national programmes and EU level initiatives such as Horizon 2020, and align with the Strategic Implementation Plan of the European Innovation Partnership on Active and Healthy Ageing (EIP AHA) [OO5].

The intervention logic underpinning the implementation of the AAL2 Programme was updated as part of the 2021 Programme Impact Assessment and is represented in Figure 1. It sets out the relationships between the objectives, inputs and activities, and defines outputs, short- and medium-term outcomes and impacts. Programme governance and management The AAL2 Programme is set-up and governed by the Participating States via a dedicated implementation structure, the Ambient Assisted Living Association (AALA) set-up as an international not-for-profit association under Belgian law (AISBL). The highest decision-making body of the AALA is the General Assembly (GA), which comprises representatives from all Participating States who enjoy the same voting rights. Members of the GA are typically national funding authorities (ministries and/or funding agencies). If more than one agency for the same country is present in the GA, a representative (bearing the country vote) must be designated. In total 21 countries participated throughout the AAL2 Programme’s lifecycle, 16 EU Member States, 3 Horizon 2020 associated countries and 2 third countries , . The Commission has observer role in the GA. Nevertheless, the Commission is a party to the delegation agreement with the AALA as regards the EU’s financial contribution and has a veto right over the Annual Work Programme (AWP) approval. Programme activities The AAL2 Programme addresses applied research and innovation (R&I) in the areas of active and healthy ageing systems and applications with a short to medium term time horizon and a time-to-market of 2-3 years. The Programme is designed to complement longer-term research in ICT for Healthy and Active Ageing under Horizon 2020, which focuses on advanced R&I with a time-to-market of 5-10 years. The main activity of the AAL2 Programme is launching annual transnational project calls, selecting and supporting promising transnational R&I projects that contribute to the fulfilment of the Programme’s objectives. Participation in calls is generally possible for applicants from Participating States providing financial contributions to AAL2 calls. Actual participation in AAL2 projects depends on the application success in the selection process based on excellence criteria and the national contributions available to fund participants from the respective country. In addition to call management and implementation, the AAL2 Programme established a portfolio of Support Actions (Figure 11: AAL Support Actions portfolio in Annex VI) aimed at strengthening the capacity of Programme beneficiaries and the wider AAL community. Support Actions were largely targeted at specific shared needs and responded to the challenges experienced by beneficiaries in implementing their projects, especially difficulties in bringing their products or services to the market and community building support. 2.2 Point(s) of comparison The 2014 decision stipulates that AAL2 should build on the achievements of the previous programme, the Ambient Assisted Living Joint Programme (AAL1) and address its shortcomings by encouraging stronger user participation in projects and by a more agile programme implementation. Additionally, the AAL2 Interim Evaluation, carried out in 2017, covering the first three years of operations of AAL2 (2014-16) resulted in some recommendations for the second term of the AAL2 Programme [RIE]: ·Enhance the strategic leadership of the AALA through appropriate appointments [R1IE]; ·Further intensify support for business planning and market research to strengthen commercialisation and build engagement with the community supporting active and healthy ageing [R2IE]; ·Provide focused support on key framework conditions shaping the development of the market for innovative solutions, support the integration of solutions in existing systems and encourage testing on the demand-side [R3IE]; ·Intensify communication and outreach to increase the visibility of the Programme [R4IE]; ·Secure the sustainability of the Programme for the remainder of the term, including through stabilising the number of Participating States [R5IE]. The present evaluation includes an assessment of the progress made in this respect and is following up on the points of comparison used for the interim-evaluation, broadened with additional indicators where relevant. A more detailed discussion and underpinning evidence is provided in Section 4 of this document where the different evaluation criteria are analysed. The present evaluation was informed by the above intervention logic (Figure 1) and focuses primarily on the outputs, and outcomes levels (blue-coloured parts of the framework). Data on the long-term impacts (green, red and orange parts in Figure 1) of the AAL2 Programme are still emerging. For a definitive assessment of impacts it is too early because they include both direct and indirect macro-level results enabled by the Programme and depend on the research projects’ outcomes and the uptake of the results ideally 2-3 years after project implementation. It should be recalled that AAL2 projects deliver prototypes of digital solutions which typically cannot immediately be brought to market without further product-development efforts. This combined with the fact that a significant share of the AAL2 research projects are still ongoing explains why it is at this moment in time too early to make an overall assessment of the long-term (+5 years after project completion) Programme impacts of AAL2.

3.3. How has the situation evolved over the evaluation period? 3.1 Current state of play The AAL2 Programme built on the structures and activities developed during AAL1. It has continued and expanded its portfolio of activities. An Interim Evaluation of AAL2, conducted in 2017, concluded that the Programme successfully transitioned from AAL1 to AAL2. It made good progress towards its objectives and remained highly relevant because of the growing importance of demographic change in Europe. In 2017 the Central Management Unit (CMU) went through a major restructuring after the Interim Evaluation, to align with the increasingly complexity of the Partnership and its stronger innovation drive requiring different managing and coordinating skills and competences. The Programme further expanded its activities, especially with regard to the Support Action portfolio and increased Programme visibility. The final transnational project call was launched in December 2020 and from mid-2021, the AAL2 Programme is gradually being phased out. Activities will stop completely once ongoing projects have ended. The Programme is expected to close by end 2027 at the latest. While the Final Evaluation is aimed to assess the achievements of the Programme towards its objectives during AAL2 only, in a few instances it is necessary to include information from the predecessor programme, AAL1 that ran from 2008-2013 under Framework Programme 7. Between 2014 and 2021 the Programme issued eight calls for proposals. It should be noted that the 2021 call was not initially planned but made possible following a decision by the GA. However, due to national budget limitations, not all countries were able to contribute financially to this additional call. The calls were organised according to priority themes: ·2014: Care for the future ·2015: Living actively and independently at home ·2016: Living well with dementia ·2017: Integrated packages of solutions ·2018: Smart solutions for ageing well ·2019: Sustainable smart solutions for ageing well ·2020: Healthy ageing with support of digital solutions ·2021: Advancing inclusive health and care solutions for ageing well in the new decade Over time, the portfolio of Support Actions increasingly moved to addressing the needs of the wider community involved in active and assisted living, for example, through addressing regulatory issues of relevance to the market, as well as issues concerning the ethical implication of novel solutions and concerns about ensuring data privacy. At a later stage, the Programme also developed and implemented Support Actions aimed at strengthening the wider ecosystem required for integrating novel technologies in existing health and care system in a successfully and sustainably way. During the second period of implementation of the AAL2 Programme a number of changes and improvements were introduced to further increase the relevance, efficiency and effectiveness of the Programme, optimise the organisation of calls and support successful applicants in bringing their solution to the market. They include, for example: ·The introduction of impact assessment and key performance indicators (KPIs) to help the Programme focus on delivering outcomes and impacts; ·The expansion of call themes from narrower topics to broader challenges to give more space to innovative proposals; ·The possibility to apply for Small Collaborative Projects (SCPs) to explore new approaches, or target new business sectors or stakeholder groups through smaller scale projects of shorter duration; ·Following the onset of the COVID-19 pandemic, the application window for calls was extended to 8 months to allow applicants, especially user organisations, more time to prepare; ·The reconfiguration criteria were further optimised for projects in which applicants needed to be replaced because of insufficient budget available at national level. The maximum share of uncovered budget (in terms of person months) for a proposed project to be eligible for reconfiguration was increased from 20% to 25%; ·Programme beneficiaries were required to develop a business plan for their project and were given feedback on it at their mid-term review (MTR);

·Support Actions portfolio was widened, synergies and closer collaboration with other programmes were realised. 3.2 Key Programme statistics During the period 2014-21 the AAL2 Programme funded 151 projects with a total of 705 beneficiaries and 1078 participations . The total contracted project funding volume amounts to EUR 186.1 million public funding, of which EUR 98.3 million (52.8%) were national grants, the remaining EUR 87.8 million (47.2%) were European Commission (EC) contributions . The sum of beneficiaries’ own (i.e. private) contributions amounted to EUR 77.7 million. The average contracted budget per project (including own contributions) is EUR 1.7 million; the average contracted public funding volume per project is EUR 1.2 million. Next to co-funding projects, EC contributions also cover administrative costs for a total of EUR 6.5 million and Support Actions for EUR 6.4 million over the period 2014 to 2021. Figure 2 gives an overview of six key funding indicators throughout the AAL2 Programme cycle. The total number of individual participations in projects peaked at the beginning and at the end (153 participations) of the AAL2 Programme, steadily decreasing until 2017, where a low of 117 participations was reached. However, a similar trend, with a delay of 2 years, can be observed for the different funding contribution levels, which decreased from EUR 40.9 million in 2014 to EUR 26.3 million in 2019, but then rose again to EUR 36.8 million in 2021. In these data, it is not obvious whether the COVID-19 pandemic has had an impact on project participations and funding in 2020 and 2021. More Programme data can be found in Annex VI ‘Set-up of the Programme and Additional Programme Statistics’. 3.4. Evaluation findings (analytical part) 4.1 To what extent was the intervention successful and why? 4.1.1 Effectiveness The effectiveness of the Programme is presented in terms of the progress made towards each of the three strategic Programme objectives and assessed against the linked operational objectives. Objective 1: Enhance the availability of ICT based products and services for active and healthy ageing, to improve the quality of life for older people and their carers, and help increase the sustainability of care systems The assessment of progress towards Objective 1 starts by examining how the involvement of end-users in funded projects has developed during AAL2 and presents information on output, outcome and impact, including the development and deployment of solutions and their impact on the quality of life of the older people, their families and carers. User involvement and profile User participation and user-centric development is essential to design innovative ICT-enabled solutions to the needs of the elderly people, carers and intermediaries, including the settings in which the technology is to be used. The AAL2 Interim Evaluation assessed the degree of involvement of end-users in funded projects. It found that 18% of project participants had been users, a figure that had remained unchanged compared to AAL1. The category ‘users or end-user organisations’ covers three specific classifications of users – primary, secondary and tertiary – although the three classes are not broken down (see Annex VI for definitions). The Interim Evaluation stated that, while users are well represented, their engagement was still deficient in terms of (1) breadth, i.e. stakeholders not been representative of all types of users and (2) many of them are proxy users rather than end-users themselves (e.g. associations, charities). The involvement of user or end-user organisations in new projects increased significantly since the publication of the Interim Evaluation (see Figure 3). Their number more than doubled from 26 in 2017 to 54 in 2021. Over the whole Programme period of AAL2, end-user organisations constituted approximately one quarter of all project participants (see Annex VI). End-user organisations can be further disaggregated by type of organisation (Figure 4). Around 20.8% of all end-user organisations are foundations (including charities). About 15% are care organisations such as care facilities and care services. User-involvement clearly has a central role in the development of solutions for active and healthy ageing and is a key modality of the Programme aiming at improving the adoption and relevance of the developed solutions. We conclude that the share of end-user organisations’ participations increased significantly (from 18% in AAL1 to 26% in AAL2) and that the breath of types of end-users diversified and included more actors from the health and (social) care demand-side over the second half of the AAL2 programming period. [linked to OO3 and R3IE]

Development and deployment of solutions The 8 calls for transnational projects and resulting 151 funded projects directly contribute to developing and enhancing the availability of ICT based products and services for active and healthy ageing. Measuring the actual deployment and impact of such solutions is more difficult as already explained earlier. The 2021 Programme Impact Study stated that 23 of the 38 (61%) AAL2 projects brought at least one solution or component to the market. Considering that most solutions require 2-3 years to mature after the end of a project before they can be launched, this is a positive result. Numbers provided by the 2019 and 2020 Programme Impact studies also demonstrate a steep increase in solutions in the market, with 24 of 85 projects launching a solution or component . The 2013 Commission Impact Assessment proposed a target of 25% of all running projects that should reach the market by 2020. The above collected evidence a exceeds this target and shows that AAL2 developed solutions have increased chances for market success with shorter time to market cycles although it is still too early to generalise this conclusion. Once the products and services are on the market, they are adopted by end-users. The number of users could thus be seen as an indicator for deployment and use of the AAL solutions. The revenues generated by the launched products/services represent an increased commercial income for the industrial base in Europe and give an indication on the impact of the programme at the level of markets and solutions. The 2021 Programme Impact Study presents information about the revenue generated by digital solutions developed in the projects, the number of individual end-users using the solution after one year and the number of paying customers. The average total revenue for projects with positive revenue was EUR 243 875 per year for products or services, and about EUR 50 000 for components . These figures were slightly higher for projects included in the survey presented in the Programme Impact Study reports published in 2019 and 2020. However, these two surveys largely related to projects that had been launched under AAL1 and their solutions had more time to mature. The average total number of individual end-users per solution in one year was 1 855, an increase of around 30% compared to the numbers reported in the 2019 and 2020 Programme Impact Studies. The average total number of paying customers per solution in one year was 771, compared to approximately 1 000 paying customers using solutions developed by the projects included in the survey in the two previous years (including AAL1 solutions which had more time to mature). Reaching a market of paying customers can be seen as an important condition to increased availability of digital products and services (strategic objective 1), both on the side of industry as well as on the side of the users of the products and services (older adults, informal carers, care organisations, insurers, public authorities). It is not useful to compare these outcomes against the outcomes of other public-public partnerships, given that the AAL2 Programme is special in addressing an immature, fragmented and highly regulated market. However, compared with the internal benchmark of AAL1 (using the 2019 and 2020 Programme Impact Studies) numbers of solutions commercialised across (and beyond) Europe, numbers and type of customers and end-users reached per year show a positive trend and tend to outperform the 25% target set in the 2013 Impact Assessment. [OO2 & RI3] Impact on quality of life of older people and carers The 2021 Programme Impact Study also presents initial findings on the impact of the developed and deployed solutions on the quality of life of older people. Only beneficiaries participating in the 38 finished projects with a full product or service already launched on the market were asked to respond to this question. Of these respondents: 9/13 agree that AAL solutions helped older people to live independently in their own homes; 8/13 think they helped them remain in better mental condition; and 6/12 agree that they help them succeed in independently interacting with their network. Conversely, 5/12 thought that AAL solutions helped the older people to feel less dependent on others for daily tasks and 4/12 continue participating in common activities with their network. The 2021 Programme Impact Study also asked beneficiaries about the impact on carers. Of the 13 respondents, the following numbers agree with the following statements: 9/13: care visits are more relevant thanks to automated risk detection tools; 8/13: carers have more time for qualitative interaction with the older adult; 8/13: carers feel the effect of being less burdened; 7/13: carers in the formal sector feel they save at least 5% of costs overall thanks to better planning and coordination; 5/12: carers spend at least 15 minutes less on looking for information or coordinating with other carers. Given the sample size of completed research projects (n=38/151) at the time of the survey and consequently the lower number of respondents to this question (n=13), it is difficult to provide a judgement. Nevertheless, these first findings, confirm the potential of the developed AAL2 digital solutions in support of active and assisted living. The 2021 Programme Impact Study also provides insight into the impact of AAL2 on raising awareness for solutions in the field of active and assisted living. 85% of formal carers and 91% of older people responding to the survey agree with the statement that the AAL Programme helped raise awareness of the needs of older people and the opportunities associated with solutions for healthy living and well-being among formal carers as well as older people and their networks. In the interviews of the supporting study, representatives of user organisations explained that the organisation and (local) financing of care as well as expectations from older people and their families strongly shape the demand-side of the innovation market. Circumstances vary substantially between and within countries. In some countries, end-users are largely expected to be individual older people or families while in other countries formal care organisations play a more substantial role in shaping the demand-side. This implies that supporting solutions might be paid out of pocket by individuals while in another contexts they are purchased by care organisations. The demand-side may also involve public funders and is shaped by different degrees of regulation. These national differences in health and care ‘market models’ explain to some extent the difficulties faced by innovators during market launch and afterwards when trying to keep their product or service on the market. However, interviewees also stated that expectations and thus markets have been changing and are continuously evolving, with technology becoming more familiar and acceptable, and most people wanting to live at home as long as possible. This was also noted in countries that had a tradition of older people moving into residential care, which was understood as not being sustainable in the future. Stakeholders and beneficiaries also noted that while there are still obstacles to marketisation due to the structure of the market, there is significant potential in developing and promoting innovative solutions for active and healthy living. [linked to R3IE] It is clear from these assessments that outcomes and impacts are yet difficult to measure. The consecutive Programme impact studies have made progress in providing better data, but available data are still limited in scope and detail. However, the limited data currently available shows an upward trend in AAL solutions brought to market and demonstrate some impact on end-users while qualitative findings suggest that the market conditions are evolving and that the full potential for impact has not yet been fully realised. Objective 2: Create/maintain a critical mass of trans-European applied research, development and innovation for ICT-based products and services for ageing well, in particular involving SMEs and users To measure progress made towards Objective 2, the Interim Evaluation examined KPIs such as the number of countries involved in AAL2, national commitments made, budget utilisation, the number of organisations participating in each call and the number of projects funded. This approach was also used in the Final Evaluation of AAL1, which had referred to ‘the presence of a sufficient number of actors, sufficient cooperation, and sufficient total R&D and innovation activity to initiate a self-sustaining, productive and viable research environment’ . The AAL2 Interim Evaluation concluded that ‘indicators for these elements were mostly moving in an undesirable direction’ and that ‘both large countries and key innovators have been lost from the Programme’, ‘too few projects start[ed] from validated user insights or had identified customers for the solutions proposed . The assessment of progress towards Objective 2 builds on the above KPIs updated and complemented with other output and outcome-related indicators. [linked to OO1 & R5IE] Number of proposals and projects

As shown in Annex VI (Figure 14: Number of proposals and individual applicants per year) the first years of AAL2 showed a more than 50% decline of proposals from 97 in 2014 to 42 in 2017, the period assessed by the Interim Evaluation. From 2017, the number of proposals increased again to 106 in 2020, outperforming the 2014 figure. In 2021 (limited additional call), the number of proposals was 85. Therefore, the negative trend during the first half of AAL2 did not continue during the second half of the Programme and was reversed. The same observation applies to the number of projects starting during AAL2 (Annex VI Figure 13: Number of projects per application area, per year). Between 2014 and 2017 the number of newly started projects decreased from 20 to 14 per year. Yet in 2018 this number increased to 21 and remained at a high level until the end of the Programme. The decrease observed during the Interim Evaluation was thus reversed in the second half of AAL2 [linked to OO1 and R5IE]. Proposals and projects per type of organisation Objective 2 makes explicit reference to SMEs and end-users as priority target groups of the Programme. Over the entire period of AAL2, 42.1% of applicants were SMEs and 24.9% were end-users (Figure 18: Share of individual participations in proposals by type of organisation) . While the number of SMEs among applicants fell from approximately 330 in 2014 to 152 in 2017, the number recovered and stabilised at about 250 towards the end of AAL2. There was a strong increase in the number of end-user organisations among applicants since 2017, which rose from 69 in 2017 to 226 in 2021 (Figure 3: Number of project participations per type of organisation, per year). In terms of participation of SMEs and end-user organisations in projects, the respective shares were 40.3% for SMEs and 26.3% for end-user organisations (see Annex VI Figure 19: Share of individual participations in projects by type of organisation). In summary, it can be stated that the priority target groups of the Programme, SME’s and end-users were well represented in the number of applications received, as well as in the projects selected for funding. [linked to OO3, OO4 & R5IE] Geographical distribution of applicants and project participants The geographical spread of individual participations in proposals was uneven, with some countries participating significantly more strongly in proposals than others. These can be grouped as follows: ·Group 1 – Countries with ≥ 500 individual participations in proposals: oNetherlands (731), Switzerland (599), Austria (591) ·Group 2 – Countries with 201-500 individual participations in proposals: oRomania (414), Italy (396), Portugal (282) ·Group 3 – Countries with 101-200 individual participations in proposals: oBelgium (200), Spain (192), Slovenia (169), Poland (143), Hungary (135), Denmark (134), Norway (122), Cyprus (116) ·Group 4 – Countries with ≤ 100 individual participations in proposals: oUK (72), Luxembourg (40), Canada (39), Sweden (17), Ireland (12), Taiwan (9), Israel (6) With few exceptions (notably Spain and the UK, with the latter not participating in calls after 2015), the difference in the distribution of applications between countries did not change substantially during AAL2. Individual participations in projects show a similar pattern of distribution between countries:

·Group 1 – Countries with ≥ 150 individual project participations: oAustria (166), Switzerland (152), Netherlands (166) ·Group 2 – Countries with 51-100 individual project participations: oItaly (97), Romania (88), Belgium (84), Portugal (63) ·Group 3 – Countries with 11-50 individual project participations: oSpain (45), Denmark (42), Hungary (33), Norway (33), Poland (30), Cyprus (22), Slovenia (21), Luxembourg (11) ·Group 4 – Countries with ≤ 10 individual project participations: oUnited Kingdom (10), Taiwan (10), Ireland (7), Sweden (6), Canada (5), Israel (2) The map below (Figure 5: Geographical distribution of project participations per country) illustrates the distribution of project participations across the participating countries. Source: Combination of AAL administrative monitoring data with Bing Maps. Key learnings identified by participants Whereas the above statistics mostly relate to the activities supported by the AAL2 Programme, an analysis of effectiveness also needs to consider the outputs and outcomes of those activities. A key indicator in this respect are the learnings that have been identified by participants. The 2021 Programme Impact Study provides relevant data on four dimensions of learning relating to: (1) user needs; (2) operational knowledge; (3) markets; (4) investors. For each of these dimensions, a survey asked beneficiaries whether they felt their knowledge had increased (a) as a result of participating in AAL2 (targeting all 91 respondents) and (b) as a result of participating in the AAL2Business Support Action (with 33 survey participants responding to this question). With regard to (1) user needs, 82% of 91 respondents indicated that their knowledge had increased as a result of collaborating with users and adapting to user needs; 47% agreed that this had resulted in an increase in user acceptance. Out of 33 respondents who participated in the AAL2Business Support Action, 76% confirmed that their ability to identify user needs and develop user profiles had improved to a moderate or great extent as a result of participating in AAL2Business. 73% stated that participating in AAL2Business had improved their ability to involve users in developing a product or service to a moderate or great extent. In terms of (2) operational knowledge, 66% of 91 respondents agreed that their knowledge had increased on technological aspects. Moreover, 32% and 30% acquired new knowledge on aspects of marketing and regulatory aspects, respectively. While gains in knowledge on marketing and regulation were lower than on technological aspects, these were not central to the calls. However, 42% of 33 respondents stated that participating in Support Actions improved their knowledge on intellectual property rights to a moderate extent. In relation to knowledge gained on (3), the market for active and assisted living through regular calls: 35% of 91 respondents confirmed that they had gained new insights into the consumer market relevant to AAL solutions; 30% agreed they had learned about the adaptation to geographically different markets; 28% agreed that participation had strengthened their ability to assess commercial viability; 14% agreed they had gained credibility and improved their ability to provide evidence to convince investors. Strengthening the marketisation skills of beneficiaries had been identified as a key area for support and thus became the focus of the AAL2Business Support Action. This has had an effect: of the 33 respondents who had participated in the Support Action, 61% agreed that their participation had improved their ability to develop a commercialisation strategy; 75% agreed that participating had improved their ability to develop a business model. [linked to R2IE, R3IE and OO2] Network creation Another indicator for measuring progress towards Objective 2 is the degree to which AAL2 has helped to develop networks among relevant actors in the Participating States. The survey conducted for the 2021 Programme Impact Study shows that for the 91 respondents, 68% stated that they benefited from the collaboration networks established during the project. Of the 56 who had participated in the yearly organised networking event, the AAL Forum, 89% stated that the Forum helped them to broaden their network in relevant fields.

In interviews as part of the supporting study, beneficiaries noted that participating in networks and having the opportunity to find new partners for collaboration was a key reason for their participation in AAL2. They specifically welcomed the opportunity to collaborate across borders and within European networks and highlighted the benefits from those collaborations for inspiration, learning and improvement. This was confirmed by National Contact Points (NCPs) and stakeholders who also suggested that these opportunities may have been especially relevant for beneficiaries from smaller countries, or from countries where the market for AAL2 was still underdeveloped compared with other national/regional markets and systems. Opportunities for collaboration were also fostered by the introduction of SCPs as a new instrument that allowed beneficiaries to explore new ideas at a small scale over a shorter period of time within a participatory, collaborative framework. [linked to OO1, OO3, R4IE] Objective 3: Leverage/maintain private investments and improve industrial conditions for industrial exploitation by providing a coherent framework for developing European approaches and solutions including common minimum standards that meet varying national and regional social preferences and regulatory aspects The market for active and healthy ageing solutions is fragmented, highly regulated and facing a complex demand-side. There are many obstacles to new solutions being launched and performing successfully on the market. In a similar vein as the Interim Evaluation, the assessment of progress towards Objective 3 focuses on support for business planning and commercialisation, the involvement of large industry partners, and the level of private investment leveraged by the Programme. The Interim Evaluation suggested that more involvement of large industry partners is needed to scale-up solutions and create significant impact on both the market and end-users. It also concluded that many beneficiaries were insufficiently prepared for marketisation, that business planning was insufficient and that the AAL2 as a whole lacked an integrating framework for individual project results. Support to business planning and commercialisation Previous evaluations emphasised the importance of improving participants’ business orientation, market knowledge and commercialisation skills, particularly for strengthening their capacity to develop a business plan, find investors and bring solutions to the market. Already during AAL1 the programme invested in developing a portfolio of Support Actions, next to the project calls. This comprehensive programme-driven approach was strengthened and expanded significantly during AAL2, with a dedicated set of Support Actions specifically addressing the above mentioned needs. Especially AAL2Business and the Market Observatory actions provided project beneficiaries with the skills and knowledge required to bring their product or service to the market. This was seconded by improvements to the project call procedures, requiring applicants to include a business strategy as part of their proposals. Interviews and the available data from the beneficiary survey demonstrate that the Support Actions were highly valued by participants and contributed to feeling more confident and becoming more successful in navigating the market, engaging with the demand-side, exploring and integrating user expectations, and developing business and commercialisation strategies. Beneficiaries, NCPs and other stakeholders interviewed were highly appreciative of the Support Actions, which were rated as exemplary and good practices to follow. This finding is further confirmed in the 2021 Programme Impact Study; 61% of respondents who participated in Support Actions (n=33) stated that their participation had improved their ability to develop a commercialisation strategy and 75% agreed that the Support Actions had improved their ability to develop a business model. Beneficiaries’ own contributions and private investments into the Programme Over the entire AAL2 Programme period, own funding contributed by participants in AAL projects amounted to EUR 77.7 million, which equalled 41.7% of the combined EC and national funding (sum of public programme contributions). Hence, this sum can be seen as the amount of private investments being directly triggered by the Programme. The study supporting this final evaluation also assessed private follow-up investments attracted by the Programme beneficiaries by using the platform Dealroom . Dealroom provides data on investments into companies by various business organisations such as banks, investors or business angels. While it is not possible to link this investment data directly to specific AAL2 projects, they may provide interesting contextual information. From a dataset that contained the URLs (Uniform Resource Locators) of 703 AAL2 participants , we could match 205 participants (approximately 30%) to data entries included in the Dealroom database. Dealroom identified investments being made in 35 out of those 205 organisations (approximately 17%) between 2014 and 2020. In total, these investments amounted to around EUR 374 million , with one single company, MindMaze from Lausanne, receiving EUR 307.7 million. Ten more companies received investments of more than EUR 1 million. The remaining 24 companies stayed below the EUR 1 million threshold. The 2020 edition of the Programme Impact Study looked at the investor events that had been organised by the AALA in 2017 and 2018. In total, these five events attracted 27 investors and an equal amount of companies, pitching 26 different solutions (with some of the solutions being pitched multiple times, bringing the total number of pitches to 37). While no direct investment being secured as a result of the event could be identified, the authors of the study supporting the final evaluation passed the list of pitching companies through the Pitchbook database. Out of the list of 27, Pitchbook contained records on 13 of these. Out of those 13, the database had information on investments, mergers and acquisitions for 7 companies, with mostly undisclosed amounts. However, the authors noted that none of the investments made, suggest a direct link with the AAL investors’ events. Involvement of large industry Although large enterprises were not defined as a core target group to be addressed by the Programme, the Interim Evaluation noted that the development of a new market requires not only agile and innovative SMEs with a strong focus on the user perspective, but also large industry partners that can contribute the expertise and market power to launch and establish new solutions in the market. When looking at Figure 15: Share of project participations by type of organisation, per year, Annex VI), it becomes clear that the involvement of large industry players remained relatively low and did not change over the entire duration of the Programme. Overall, the AAL2 Programme has achieved positive results at the output and outcome levels within its remit. While it supports beneficiaries in overcoming market barriers, it is not in a position to remove these barriers entirely. Data on long-term / macro-level impacts are limited mostly as it is still too early to conduct a full impact analysis of the Programme. A key issue that needs to be taken into account when considering the effectiveness of AAL2 is that it operates in a fragmented and still immature market and that many of the barriers to successful deployment and commercialisation of digital solutions for active and healthy ageing lie outside the scope of the Programme. Compared with AAL1 (as an internal benchmark) there was a positive trend in the numbers of solutions launched, and individual end-users reached. A beneficiary survey and stakeholder interviews confirmed the potential for positive impacts of solutions on the quality of life of older people and their carers. A strong feature of AAL2 is its portfolio of Support Actions that have proved to be an effective accompanying measure to the annual calls for transnational projects. By supporting beneficiaries to learn from each other, understand the market, and develop business models and commercialisation strategies, Support Actions provide high additional value to the Programme and towards achieving its objectives. [linked OO3, OO4, R2IE & R3IE] 4.1.2 Efficiency The efficient implementation of the Programme, includes the governance structure of AAL2, the performance of the CMU, the administrative burden at Programme and project level, and its budget utilisation. In addition the flexibility of the Partnership towards adjusting its objectives, activities and resources to changing circumstances has been assessed. Implementation and governance structure The voluntary nature of the Article 185 initiative and the possibility of having varying levels of commitment (in terms of financial contributions and/or content-related engagement) was seen as helpful to kick-start the Partnership. It was a way of getting the initiative started in a field that was still largely underdeveloped and to which countries gave uneven priority. By building an ‘alliance of early movers’ the Programme was tapping into the energy and enthusiasm of those dedicated to promoting digital solutions for ageing well. The fact that countries were not obliged to commit a specific financial contribution or other types of engagement for the entire Programme duration, lowered the threshold for them to join. However, this in-built flexibility proved to be challenging in the longer term. The decrease in funding commitments which occurred during the first half of the AAL2 Programme (Annex IV, Table 5: Budget committed per country, per year) challenged the financial sustainability of the Programme at mid-term . Interviewees indicated that the possibility and/or willingness of NCPs to engage in developing a joint strategic agenda varied. A few funding agencies may have seen their role primarily in following due process in managing project funding, with less attention given to providing strategic direction. Some of them noted that they had tried to align national strategy, but they had not been able to convince their political masters to support AAL as strongly as they wished. This meant that the CMU had to play a strong role in driving strategy development while encouraging NCPs to engage in the process.

It has also been noted that the EC has a different role in the governance structure of the Programme compared to the Participating States. While the Programme is largely governed by the Participating States who form the main decision-making body, it is also a Partnership between the Participating States and the EC as per Decision No 554/2014/EU. The EC contributes 50% of public funding, yet in the GA the EC has observer status only. The only effective mechanism for the EC to influence the strategic direction of AAL2 is during the approval process of the AWP. Some interviewees therefore suggested that the governance structure is weighed against the EC. However, in general, the assessment of NCPs and other stakeholders about the functioning of the governance structure of the Programme was positive. A modus operandi was found that appeared to work well for the various decision-making processes in place and almost all NCPs were satisfied with the strategic direction and processes of engagement during AAL2 Programme execution. [linked OO1 & R1IE]. Performance of the CMU The findings of the study supporting the final evaluation confirm the assessment of the Interim Evaluation that the CMU operates effectively and at a high level of competence. Based on the insights from interviews and desk research, the concerns about the lack of strategic capacity expressed in the Interim Evaluation, is not confirmed. The CMU went through a major restructuring after the Interim Evaluation, to align with the increasingly complexity of the Partnership and its stronger innovation drive requiring different managing and coordinating skills and competences. The vast majority of individuals interviewed lauded the leadership, commitment and energy of the CMU and saw its performance as a key strength of AAL2. NCPs and other stakeholders provided highly positive feedback with respect to the performance and role of the CMU in developing the Programme, especially during the second phase of AAL2 after a change in leadership. It was also stated that the level of dedication to its mission distinguishes AAL2 from the other R&D programmes. This positive assessment was also reflected in the MTR satisfaction survey conducted by the AAL CMU in 2020 . The CMU implemented a number of changes to the management of the Programme as outlined in ‘Section 3.1 Current state of play’. These have improved its efficiency and focused the Programme on maximising the outputs, outcomes and impacts. Stakeholders described the management approach as proactive, flexible and inspirational and noted that this approach was specific to AAL2 and contrasted to their experience of other (Horizon 2020) programmes. In this respect, the Partnership seems to have enjoyed an unusual degree of managerial freedom that was used to its advantage and appreciated by its members. [linked to 001 & R1IE] Administrative burden The public consultation on the Article 185 initiatives conducted in 2017 concluded that the administrative burden on applicants was higher compared to projects that were funded by one country only. Yet it also concluded that projects funded by Article 185 partnerships were less administratively burdensome than Horizon 2020 projects insofar as applicants were already familiar with national funding rules. Beneficiaries interviewed noted that the use of national funding rules simplified the process for them, which was especially relevant to SMEs, although national funding rules and procedures varied significantly in complexity. More critically, interviewees stated that they needed to report separately to the EC and their respective NCP, often in two different formats, which was consistently felt to be an added burden. Financial matching and budget utilisation As an Article 185 initiative, the AAL Programme combines national project (co-)funding with a centrally organised selection process based on three core criteria. This constellation requires a complicated matching process to make the best use of the available funding and minimise the risk of underspending. Irrespective of its excellence, an application can only be funded if all Participating States involved as national funders still have sufficient budget available. If not, applicants whose country has run out of national budget, can be replaced by other participants from countries that still have budget available . This matching process was introduced to address problems in budget utilisation resulting from the combination of applying national funding rules together with a central evaluation process. All projects invited for reconfiguration during AAL2 (47 in total) succeeded in reconfiguring their proposals and signed grant agreements, with the exception of one project in Call 2020. Though, the matching process requires substantial additional administration and thus introduces new challenges. In the interviews, NCPs and beneficiaries reported difficulties in reconfiguring projects as this required them to redistribute tasks within the consortium or identify new beneficiaries that could execute specific work packages. Interviewees agreed that the process was manageable but created additional challenges. The MTR satisfaction survey confirmed that the majority of beneficiaries involved in project reconfiguration were satisfied with the outcome. In total, the budget utilisation under AAL2, defined as the ratio between total budget committed by Participating States and annual budget contracted over the Programme period was 79.4% . Budget utilisation varied between years, with the utilisation rate being as low as 65.1% in 2019 and as high as 97.2% in the last call in 2021 (see Figure 6). The variation of the project budget utilisation rate was even larger when comparing different countries (see Figure 7). While Switzerland, Austria and the Netherlands contracted more funding than originally committed, Israel, Luxembourg, Belgium and the UK contracted significantly less than 50% of what they had originally committed. There are a number of reasons that can explain the under-utilisation of the committed budget. First, there was a lack of absorption capacity in some countries illustrated by a low number of proposals submitted and/or a low number of highly ranked proposals in those countries . This led to 28.7% of the committed budget not being used. Second, some proposals could not be funded because at least one country had already depleted its committed budget. This accounted for 8.2% of the budget underspend. However, thanks to the possibility of reconfiguring projects and some national budgets being increased, an additional 18.8% of the originally committed budget could still be contracted. [linked to OO1, OO4 & R5IE] Cost-effectiveness and administrative overhead Figure 8 shows that the overall EU AAL2 Programme commitment amounts to EUR 108,6 million, which is significantly lower than the initial, maximum estimated Union’s financial contribution of EUR 175 million at set-up of the Programme. Article 2 of legal base (Decision No 554/2014/EU) stipulates that the annual financial commitment of the Union to the AAL2 Programme shall not exceed the annual financial commitment to the AAL2 Programme by Participating States. The decreasing national funding commitments was an element of attention raised at the AAL2 MTR. It resulted partly from fewer countries being involved then initially planned and partly due to decreasing contributions from those that remain. In addition, some countries that have signed participation agreements don’t always actively contributing to yearly transnational calls e.g. only 16 countries contributed to call 2016 (see also Annex VI Table 5). However, this negative trend was reversed in 2018 with increasing national funding commitments peaking at almost EUR 21 million commitments for the final Programme year 2020.

The administrative costs for the Programme are fixed at maximum 6% of the EC contribution. In absolute terms, this means that between EUR 0.8 million and EUR 1.2 million was used per year on administrative overhead resulting in EUR 6.5 million in total between 2014 and 2021. While it is difficult to compare the percentage of administrative overhead of AAL2 to that of other Article 185 partnerships such as ECDTP2 or Eurostars , for example, it is important to note that in absolute terms, it is significantly smaller than those of other Article 185 partnership programmes. In view of the strong performance of the CMU during the final years of AAL2, this means that the budget has been used highly efficiently . Spending on Support Actions under AAL2 was also very moderate, constituting 5.9% of the total EC contributions over this period. Support Actions were considered effective in supporting the Programme objectives and cost-effective in terms of spending. [linked to OO1]. Flexibility in adjusting objectives, activities and resources to changing market and policy needs There are numerous examples of adjustments and improvements made to the AAL2 Programme that demonstrate the flexible and forward-thinking approach taken when dealing with challenges and responding to stakeholder feedback. In terms of its strategic direction, the CMU and AALA have positioned themselves as a ‘learning organisation’ giving emphasis to monitoring, assessing and improving the implementation of the Programme. To this end, three Programme impact studies were commissioned that made a strong and convincing effort to inform improvements to AAL2. As part of this evidence-informed learning process, the definition of the objectives and the intervention logic were sharpened and changes were made to many dimensions of the Programme. Interviewed stakeholders consistently stated that the CMU was highly adept at listening to and using feedback from stakeholders and the wider community to improve the Programme. This agility was also demonstrated during the COVID-19 pandemic with adjustments made to call procedures to encourage participation and the AAL Forum moving to an online format. In terms of responsiveness to policy needs, the challenges faced by AAL2 and by innovators in the more widely field of active and assisted living, were explored by introducing an Ecosystem Support Action. This approach was developed by involving a large number of stakeholders and by systematically exploring the many factors that determine the success of novel solutions in the market. While the Programme has not been able to influence its market environment to an extent that would bring about significant improvements in the swift uptake of novel solutions, the challenge of integrating innovations into existing, complex and highly variable health and care systems is now much better understood. This way, AAL2 has laid some of the groundwork for future action and initiatives that aim to address demographic change and to improve the sustainability of European health and care systems. 4.1.3 Coherence Coherence with Horizon 2020 and other EU and national policies (programme level) AAL2 continues to occupy a unique position within the European policy landscape. It is well aligned with general EU policy objectives and acts as a bridge between research and market-oriented programmes. Its three objectives are well aligned with the overall objective of Horizon 2020 to ‘contribute to building a society and an economy based on knowledge and innovation across the Union by leveraging additional R&D&I funding’ . This applies in particular to Societal Challenge 1 on ‘Health, demographic change and well-being’, which includes ‘active ageing and self-managing disease’ as one of its broad lines of activities. Under this Societal Challenge, a number of calls and topics were launched to support the development of technologies for active and healthy ageing. Most notably, Work Programme 2018-2020 included a call on ‘Digital transformation in Health and Care’, consisting of 21 individual topics and a funding volume of approximately EUR 100 million in each of the three years. While there is a thematic correspondence between AAL2 and some of the activities funded under Horizon 2020 Societal Challenge 1, a detailed analysis shows that the respective calls and topics are complementary and synergetic to AAL2. By way of example, the topic ‘SC1 HCC-08-2020: Scaling up innovation for active and healthy ageing’ makes explicit reference to AAL2 and encourages the involvement of AAL2 partners . Also more generally, AAL2 is distinct from most research funded under Horizon 2020 due to its specific focus on innovation and SMEs, and its market orientation . This complementarity was already raised by the public consultation on Article 185 initiatives conducted in 2017, in which 64% of AAL2 respondents consider the Programme to be coherent with other EU initiatives . The collaboration and communication with other programmes, such as the EIP AHA and Joint Programming Initiative ‘More Years Better Lives’ (JPI MYBL), have been deepened during the second phase of AAL2, implementing the recommendations of the Interim Evaluation. The joint organisation of the one-week-long AAL Forum Online Event in 2020, with almost 1.000 international participants, is a tangible result of this collaboration . The Programme has also extended its outreach and collaboration with other EU institutions. These have been mainly in an advisory role, such as supporting informal working groups within the EU Parliament and deepening ties with different Commission Directorates-General (DGs). To further strengthen its external coherence and policy alignment, the Programme has expanded its advocacy role within Europe and vis-à-vis Participating States in raising awareness for the potential of innovative solutions and the need for transformative change and has contributed to setting the agenda in this policy field. At national level, the AAL2 Programme demonstrated complementarity in those participating countries, in which AAL2 provided the only opportunity to engage in this particular field (as noted by NCPs during interviews). The possibility to contribute to Programme activities in line with national capacities and needs, enabled the Participating States to work together constructively. This potentially helped those countries in which markets were small to shape the Programme according to their needs (here referred to as ‘smaller’ countries). NPCs interviewed for this evaluation noted that the AAL2 Programme was particularly valued by a number of smaller countries. For those countries, this was an opportunity to establish and develop the agenda for active and healthy ageing both domestically and within the EU. Finally, it should be stated that the AAL2 strategy is aligned with the United Nations (UN) Sustainable Development Goal 3 to ‘ensure healthy lives and promote well-being for all at all ages’ . Coherence with Horizon 2020 (participant level) To assess the degree of involvement of AAL2 participants in Horizon 2020 the study supporting the final evaluation analysed the Horizon 2020 centrality score of 699 AAL2 participants . A centrality score refers to the degree of involvement in Horizon 2020 (based on number of participations), compared to all other Horizon 2020 participants. It is measured in percentiles. As can be seen in Table 1, 5.6% of AAL2 participants were among the top 1% Horizon 2020 participants during the same period of time. This number is particularly high for universities and higher education institutes with 22% of them belonging in the top 1% of Horizon 2020 participants. To a lesser degree, this applies to research institutes, with 8% of them falling into this category. In contrast, among private companies and other types of organisations, this share is 0%; for public bodies, it is 1.1%. 56.2% of AAL2 participants have never participated in Horizon 2020, with that number being as high as 62.8% for private companies and 82.8% for public bodies.

These centrality scores show that AAL2 manages to bring together top research performing bodies in Europe with private companies (mostly SMEs) and end-user organisations that do not typically participate in Horizon 2020. This also means that there is a strong complementarity between the AAL2 Programme and other activities under Horizon 2020. [linked to OO5 & R2IE]. All AAL2 participants Universities & higher education institutes Private companies (including SMEs and large enterprises) Public bodies Research institutes Other Centrality within H2020 Number Share within category of AAL participants Number Share within category of AAL participants Number Share within category of AAL participants Number Share within category of AAL participants Number Share within category of AAL participants Number Share within category of AAL participants Top 1 % 39 5.6 % 31 22.0 % 0 0.0 % 1 1.1 % 7 8.0 % 0 0.0 % Top 10 % 127 18.2 % 64 45.4 % 24 8.2 % 4 4.3 % 29 33.3 % 6 7.1 % Top 50 % 235 33.6 % 84 59.6 % 72 24.6 % 13 14.0 % 45 51.7 % 21 24.7 % Bottom 50 % 71 10.2 % 8 5.7 % 35 11.9 % 3 3.2 % 11 12.6 % 14 16.5 % No H2020 participation so far 393 56.2 % 49 34.8 % 184 62.8 % 77 82.8 % 31 35.6 % 50 58.8 % Total 699 141 293 93 87 85 . Internal Programme coherence In terms of internal coherence, the AAL2 has made good progress in clarifying and conceptually developing the role of each of the Programme components and creating synergies at many levels. This relates to the refinement of the Programme calls as well as the Support Actions that are well integrated and designed to support the three objectives of the Programme. Support Actions were developed to run alongside the R&I projects funded by the Programme. They were based on insights gained from analysis and progress monitoring that helped identify challenges and opportunities. Some Support Actions were already present in AAL1 and were intensified during AAL2, while others have been added in response to analysis and changing policy needs (Figure 11: AAL Support Actions portfolio in Annex VI). Support Actions expanded from primarily offering opportunities for networking and exchange to developing targeted interventions aimed at specific skill and knowledge gaps. Stakeholders interviewed for this evaluation noted that there was significant learning involved in developing and adjusting Support Actions, as well as a willingness to listen to beneficiaries and expert advice, and to embrace novel ideas for improvement. Opportunities were used to apply the management tools of the project funding to identify challenges and offer tailored support, for example by using projects’ MTRs to identify weaknesses in the projects’ approaches to marketisation and provide targeted coaching to develop project beneficiaries’ business skills (AAL2Business). AAL2 also broadened the target audience from providing support to individual project teams to serving and developing the wider AAL community. Project funding was most successful in relation to achieving the first two objectives associated with bringing about innovative solutions and creating a critical mass of R&D&I. However, the third objective of shaping the market conditions for such innovations was correctly understood as being much harder to achieve and thus benefiting from specific support. During AAL2 this increasingly involved activities aimed at shaping the market, by building and engaging with its demand-side (e.g. by involving regions as purchasers and providers of health and care). It included promoting an understanding of the need for a wider approach to ecosystem building for European societies to capitalise on the potential of innovation in this sector. On the basis of the above, it can be stated that the programme design of AAL2 has created synergies between its various activities and between individual projects. 4.1.4 Transparency and openness Overall, AAL2 has demonstrated that it is open to new members participating in the AAL2 Programme and to new project beneficiaries. AAL2 has developed transparent processes for consulting stakeholders and identifying priorities in each of its core activities. This includes the preparation of calls for proposals, the development of Support Actions and the organisation of networking and outreach activities. The decentralised management of the Partnership, more specifically using national funding rules and providing NCPs, also contributes to the accessibility for SMEs. [linked to OO3 & R5IE]. Openness towards new participants and mechanism to involve new members and a broader set of stakeholders The underlying Article 185 initiative procedures allow countries to join and leave the Programme at any time. Throughout the AAL2 Programme lifecycle interested countries could apply to become member of the AALA. The AAL2 Partnership provides the possibility for flexible membership (‘variable geometry‘) for those countries interested and also allows for varying degrees of financial commitment to the Partnership. In addition, participation is possible for non-EU countries and for countries not associated with Horizon 2020. Currently Canada and Taiwan participate in AAL2 as non-EU Member States, while Switzerland, Norway and Israel were associated countries during Horizon 2020. As a specific feature, AAL2 also engages with a number of subnational regions. During AAL2, four regions participated as members and provided NCPs: Friuli-Venezia Giulia in Italy, Basque Country in Spain, as well as Flanders and Brussels in Belgium. The possibility for regions to become a member and be more actively involved in steering the Programme was seen as an important step by NCPs and other stakeholders interviewed. Interviewees acknowledged that in many countries, regions are responsible for governing health and care systems, and involving them was therefore key to unlock the potential of novel solutions for active and healthy ageing. AAL2 reached out to engage a broad set of stakeholders across Europe. The calls under AAL2 are open to international project teams with participants from science and research organisations, companies and end-user organisations. Participation in AAL2 projects is generally possible for applicants from Participating States providing financial contributions to AAL2 calls. Actual participation in AAL2 projects varied between participating countries depending on the application success in the selection process and the national financial contributions available to fund participants from the respective country. Before the COVID-19 pandemic, the annual AAL Forum was held at different locations across Europe to allow for regional spread and invite participation from a broad range of stakeholders (not limited to AAL2 beneficiaries). Also, the challenge prize as another Support Action was open to all entries in the field and not limited to Programme beneficiaries. There were also increasing efforts to provide services to the wider AAL community including the development of guidelines and regulatory information. Processes for consulting stakeholders and identifying priorities National stakeholders are consistently consulted as members of the AALA. GA meetings provide the opportunity to all Participating States to participate in key decisions and put forward their opinions, priorities and concerns. The funding authorities serving as NCPs are represented in the GA. As these institutions are in close contact with the project participants and other stakeholders in their country, they can bring in the perspectives and needs from different national backgrounds. Decision-making in the GA follows a democratic approach (‘one country – one vote’) so that all participating countries are involved in the setting of priorities for the AAL2 Programme. This particularly concerns the selection of the thematic priorities for the yearly calls which are developed in a working group drafting the AWP. This working group is open to all interested members. Stakeholders interviewed mentioned that in practice the opportunities to engage in the setting of priorities were taken up by the different participating countries to varying degrees, also depending on the individual financial and staff capacities.

Accessibility for the enterprise sector and SMEs The AAL2 Programme is accessible for different types of stakeholders across Europe. The calls under AAL2 target cross-sector international project teams. Data on the types of participants in AAL2 projects show that about 45% of participants are from the enterprise sector. The share of SMEs among the participants of AAL2 projects between 2014 and 2021 was 35.7%. This high level of participation of enterprises and especially SMEs is reconfirming the observation from the 2017 public consultation on Article 185 initiatives. It concluded that 82% of AAL2 respondents consider the Programme to be accessible for its target group, in particular SMEs . In addition, the strong national embedding of the project funding, represented by a network of NCPs as main contact points for beneficiaries as well as the use of national funding rules, lowers the barriers to participation especially for SMEs as several interviewed stakeholders noted. Nevertheless, the market still poses considerable barriers for SMEs to scale up e.g. because of the often demanding requirements from end-users such as hospitals. Therefore, high participation of SMEs in AAL2 R&D projects does not yet directly translate into short-term economic success of SMEs in the market. 4.2 How did the EU intervention make a difference? This approach has helped to develop a new narrative of demographic change that emphasised the opportunities associated with ageing societies. It has also increased awareness of the potential of ICT-based solutions to address the challenge of demographic change among many stakeholders, including national funding agencies, SMEs and end-user organisations. The Programme proves to be particularly useful to beneficiaries from smaller and mid-sized countries and regions, as it enables them to engage with new partners and other stakeholders, and to access markets in other countries. In some countries, the Programme has inspired new initiatives, complementing and reinforcing the effects and achievements at national or regional level. [linked to OO5] Additionality in terms of thematic focus and programme design The AAL Programme is novel and unique in the sense that it brings together several topics and policy areas which had previously been separate, at least at EU level, including technology and digitalisation, demographic change, and the future of health and care systems. This approach has helped to develop a new narrative of demographic change that emphasised the opportunities associated with ageing societies. It has also increased awareness of the potential of ICT-based solutions to address the challenge of demographic change among many stakeholders, including national funding agencies, SMEs and end-user organisations. By doing so, the Programme has also managed to bring together various stakeholder groups that previously had little or no interaction with each other. While some of these interactions have taken place at national level, there is considerable added value in supporting this interaction at transnational and subnational level. SMEs particularly benefited from opportunities to explore and understand markets other than their own domestic markets. More generally, the exchange of knowledge and good practices organised through the Programme was useful for all stakeholders involved. In terms of its design, the Programme differs from Horizon 2020 not only because of its market-orientation, but also because of its integrated, holistic approach to developing the field of digital solutions for active and assisted living. The portfolio of Support Actions was a useful instrument for helping beneficiaries to go beyond the development of solutions and to launch their products or services on the market. It is unlikely that the activities resulting from AAL2, especially the transnational interactions and collaborations, would have happened without the Programme being in place. In interviews, beneficiaries noted that a key reason for participating in the Programme was the possibility of collaboration with other beneficiaries across national borders and working within European networks. As yet, there is no alternative Programme with the same focus or scope that would allow international consortia to apply for funding for their collaboration in the field of solutions for active and healthy living. Therefore, in terms of its activities and corresponding outputs, the additionality of the Programme is high. In the survey of beneficiaries, reported in the 2021 Programme Impact Study, beneficiaries stated that 12 out of 13 AAL products or services and 7 out of 10 AAL components launched would not have been brought to the market or would have generated lower revenues without the support through the Programme. Contribution towards international positioning and visibility An additional partnership evaluation criteria was assessed in the study supporting the final evaluation namely the extent to which partnerships have helped foster the international positioning and visibility of the European R&I system, as well as the level of international cooperation at partnership and project level. Five non-EU countries (Switzerland, Norway, Canada, Taiwan, Israel) participate in the Programme, three of which are non-European, demonstrating the Programme’s reputation beyond the boundaries of the EU. These five countries accounted for 27.8% of total national funding, led by Switzerland contributing 24.3% of national funding. The same countries also constituted about one fifth of the 1 077 participations in project calls (202 participations). Again, most of the non-EU participations were from Switzerland (152). Non-European countries in contrast contributed only 1.3% of national funding and 1.6% of participations (17 out of 1 077). Among European non-EU partners, the Programme helped to raise awareness for the potential of assistive technologies. An impact study from Switzerland, published in 2019, concluded that the Programme has led to increased awareness of the potential, and challenges, of assistive technologies for older people among project beneficiaries and experts . Moreover, interviewees suggested that various regions were able to learn from the Programme and its international perspective. The Swiss impact study demonstrated that Switzerland’s participation in the Programme has reached most of its objectives and contributed to improved cooperation and increased research activity in the field. This is reflected by the continuous commitment of Switzerland to the Programme: across all funding calls, Switzerland has the second highest number of project participations. In terms of support to the international positioning and visibility of the European R&I system, AAL2 has made a significant contribution to strengthening the R&I collaboration with Switzerland and, to a lesser extent, with Norway and also with Canada, Taiwan and Israel. Moreover, at international level, AAL2 was able to create a network of partnerships in the field of ICT-based solutions for active and healthy ageing, which included collaborations with the World Health Organization (unit on assistive technologies), the AGE Platform in Europe and the Canadian Institutes of Health Research. Interviews with non-European partners demonstrated that AAL2 was visible among the relevant stakeholders. A NCP noted that being involved in AAL2 increased the appetite among researchers to collaborate with European colleagues. Although there was only a limited number of participations from non-European countries, participants valued that the Programme allowed for learning from experiences in other countries. This aspect is especially relevant to countries in which the AAL sector is less developed. Moreover, as highlighted during one interview, as ageing will become increasingly problematic in regions other than Europe, the AAL2 Programme provided helpful experience and guidance for future national responses to the challenge of ageing societies outside Europe. 4.3 Is the intervention still relevant? The key challenges addressed by the AAL2 Programme, as described in Section 2.1 Description of the intervention and its objectives, remain valid and their relevance will further increase in the future. In 2021, 20.8% of the EU population was aged 65 and over; this share is expected to increase to 31.3% by 2100. For people aged 80 and over, the share is expected to increase 2.5-fold from 6.0% in 2021 to 14.6% in 2100. The ratio between working age people and older people has already been shifting, with fewer younger people supporting an increasing number of older people. This has implications for delivering and financing formal and informal care and other forms of social support for older people. The impact of these developments on European societies has also been emphasised by various recent reports published by the EC, such as the EC Report on the Impact of Demographic Change and the 2021 Ageing Report . It is therefore essential that people are enabled to live more independently, stay active for longer and that society uses available opportunities to support active and healthy living. There is no doubt that ICT-based solutions have substantial potential to support active and healthy living, despite challenges experienced in implementation and public trust. Stakeholder interviews also observed that the market for innovative solutions is growing and is attracting an increasingly large number of investors from the ICT field as well as from service and care providers (including examples of utility providers). Given the breadth of applications for such innovations (e.g. in the workplace, the health and care sector and in society more widely) there are plenty of opportunities to engage in the AAL market . This potential is recognised in several EU initiatives directed at enhancing the capacity of ICT solutions for ageing well . The European Blueprint on Digital Transformation of Health and Care for the Ageing Society promotes a shared vision of a broad range of stakeholders on how ageing well should look in the digital age. It stresses the role of technology and digital innovations for providing sustainable and integrated healthcare. The 2019 Science Advice for Policy by European Academies report ‘Transforming the Future of Ageing’ highlighted the use of technology as a key area of interest for healthy ageing . The importance of technology and digital innovation for Europe’s resilience was also emphasised in the 2030 Digital Compass , setting out a vision and targets for a successful digital transformation of Europe by 2030. These include the resilience and sustainability of health and care systems. Last but not least, the EC’s 2021 Green Paper on Ageing highlights that any European policy response to demographic change also needs to address the twin challenges of achieving the green and digital transformation of society.

The AAL2 Programme is also well aligned with several of the EU2020 flagship initiatives, such as the Innovation Union, the Digital Agenda for Europe and the Agenda for New Skills and Jobs. It is also fully in line with the most recent New European Innovation Agenda , which focuses on the development of new technologies to address the most pressing societal challenges, and to bring them on the market. Finally, it also aligns with the UN Decade for Healthy Ageing and the UN 2030 Agenda for Sustainable Development . The AAL2 Programme has been unique in this space as it brought together a focus on improving active and healthy ageing and the use of innovative ICT-based solutions. Stakeholders and NCPs noted that this unique focus has been highly productive both in raising awareness for the potential of digital technologies and in strategically developing the emerging landscape and markets. As mentioned above, AAL2 has already collaborated with other European initiatives in the field of health and ageing to create synergies and ensure the sustainability of its impacts. With the various improvements to the Programme (see Section 3.1 Current state of play), AAL2 has shown flexibility in adapting to a changing policy landscape, improving Programme effectiveness and efficiency and responding to the COVID-19 pandemic, for example. However, many of the barriers for market development referred to in Section 2.1 Description of the intervention and its objectives remain and much of the potential of ICT-based solutions for active and healthy living is yet to be realised. Several of these barriers lie outside the scope of the Programme and require larger societal interventions. These include substantial political willingness to transform health and care systems and create the environment for innovative solutions to thrive and make a more substantive contribution to active and healthy living. 3.5. What are the conclusions and lessons learned? 5.1 Conclusions The Programme continues to occupy a unique position in the field of digital solutions for active and healthy ageing, with strong relevance, EU added value and additionality As the only European funding Programme focusing on digital solutions for active and healthy ageing, the AAL2 Programme continues to occupy a unique position in the European R&I funding landscape. By mobilising technological innovation and entrepreneurship to address the consequences of demographic change and strengthen the sustainability of health and care systems, it had a pioneering role in creating and seizing the many opportunities of active and healthy ageing. AAL2 actively contributed to developing a positive perspective on ageing instead of considering ageing as a social and economic problem. In doing so, it has opened a space to bring together stakeholder groups that previously had little or no interaction, and that worked productively together to develop novel solutions and foster social innovation. AAL2 differs from most other actions within the Horizon 2020 programme because of its strong market orientation and its integrated, holistic approach towards involving end-users in both the design and implementation of transnational project calls and in the portfolio of Support Actions. It has developed a portfolio of accompanying activities that can be seen as exemplary also for other programmes. The Article 185 Partnership has enabled a group of committed countries to pioneer this initiative and launch what could be called an ‘alliance of early movers’ to promote digital solutions for ageing well, without relying on the commitment of all Member States. This has allowed more than 700 beneficiaries from 21 different countries (including five non-EU and three non-European countries) to collaborate on developing solutions that help improve the well-being and quality of life of older people. This includes their families and carers, making support and care systems more efficient and sustainable, creating business opportunities and thus strengthening the industrial base in Europe, in particular the active and healthy ageing sector. AAL2 brings together a highly relevant combination of beneficiaries, including SMEs and end-user organisations that do not typically participate in Horizon 2020 The Programme succeeded in reaching its two main target groups: SMEs constitute 40.3%, and end-user organisations 26.3% of beneficiaries. Universities and research institutes constitute another 25% of Programme beneficiaries. Analysis shows that these universities and research organisations are significantly involved in Horizon 2020, which can be seen as a marker of research excellence. The Programme therefore brings together some of the top research performing bodies in Europe with private companies (mostly SMEs) and end-user organisations that do not typically participate in Horizon 2020. In particular SMEs benefit from the strong national embedding (local contact points and funding rules) which lowers the barriers for them to participate in the Programme. In addition, the number of user organisations among beneficiaries of new projects has increased steeply during AAL2. However, as stated by the Interim Evaluation of AAL2 and the Final Evaluation of AAL1, scale is important. The Programme aimed to make a sizeable difference to the development of the market, and the lives and well-being of its target audiences. Large enterprises typically have the market power, expertise and production capacity to introduce new products at scale and pace, and therefore should be encouraged to engage in this market. Yet large enterprises, initially not defined as a core target group of the Programme, only represent 5% of Programme beneficiaries. The Programme is well managed thanks to flexible implementation methods and a strong leadership role by the CMU The Article 185 Partnership that underpins AAL2 has implications for the governance and implementation of the Programme. In terms of its implementation, the possibility of having varying levels of commitment of Participating States has proved to be an advantage in launching the initiative and pioneering its objectives. However, it also involves risks, for example if too many countries choose to opt out or reduce their financial commitments. A key strength of the AAL2 Partnership is that it allows the Programme tapping into the energy and enthusiasm of Participating States. This allowed for the development of the landscape for digital solutions and established the agenda for active and healthy ageing. It has also provided the CMU with a degree of managerial flexibility in adapting the Programme, implementing improvements and inspiring change. In contrast to the concerns expressed in the Interim Evaluation, the restructured CMU was able to provide strategic leadership during the second phase of the AAL2 Programme. Stakeholders noted that because of this proactive leadership, the Programme has been able to adjust swiftly to new insights and demands and implement a number of improvements. Both the leadership style and the governance arrangements have been judged as inclusive and transparent. A characteristic of the AAL2 Partnership is the combination of a centrally organised project selection process based on excellence criteria with a decentralised mechanism of project funding. In this, the success of each application depends on both its excellence and the availability of national funding. This mechanism adds complexity in cases where proposals have received high quality scores and funding is available for some partners involved, but not for all (if respective national budgets have been exhausted). This leads to a budget utilisation rate of just under 80% , which is sub-optimal from Union’s budget consumption perspective. However, the central project selection process can be seen as an advantage as it helps national funders to focus on projects that are competitive at European level. The Programme has made progress towards its objectives, but the complex nature of the market constrains the swift uptake of solutions AAL2 has made substantial efforts to improve the supply side of the market through the development of digital solutions and supporting their marketisation and commercialisation. The Programme has also increasingly addressed barriers on the demand-side of the markets. However, many challenges to the uptake of solutions lie outside the scope of the Programme. The market for digital solutions for active and healthy ageing is still fragmented and immature, especially on the demand-side. The demand for digital solutions is shaped by a host of factors such as cultural expectations towards older age and care, the acceptance of technology, the digital literacy of its users, cross-sectorial collaboration and the organisation of care delivery and funding within health and care systems. The demand-side is also shaped by a number of intermediaries, such as insurers and public funders. These factors in combination determine whether digital solutions are seen as acceptable and whether they will be taken up. It is therefore necessary to understand that a wider approach to system transformation and far-reaching integration is needed to fully use the potential of digital solutions for active and healthy ageing. Compared with AAL1, there was a positive trend in the number of solutions launched and individual end-users reached under AAL2. The potential of the solutions to impact positively on the quality of life of older people and their carers has been demonstrated by the survey conducted as part of the 2021 Programme Impact Study. However, these preliminary finding is based on a small number of completed projects at this point in time. In addition, beneficiaries gained substantial knowledge beyond the technological R&I expertise. Substantial gains in knowledge were achieved in relation to an increased understanding of user needs and developing business and commercialisation strategies. Beneficiaries also confirmed the positive effects of the Programme on networking and mutual learning. These effects were strengthened by deploying a coherent portfolio of Support Actions that have been widely judged as effective and supportive by beneficiaries, NCPs and other stakeholders.

Overall, it can be concluded that the AAL2 Programme has achieved positive results at the output and outcome levels within its remit. However, while it has supported beneficiaries in overcoming market barriers, the Programme was not able to remove these barriers entirely. As a consequence, the impact of the Programme is limited by the difficult market environment in which it operates with challenges remaining, particularly on the demand-side. Addressing the fragmentation of the supply side was a key motivation for the initiating the Programme. However, AAL2 has also shown how the fragmentation of the demand-side creates barriers to uptake and market success of digital innovation. 5.2 Lessons learned The following section presents key lessons learned on governance and implementation relevant to policy, strategy and operational levels. Policy level There is continued need for actions at the intersection of digital innovation and demographic change to support active and healthy living, and secure the future of health and care systems. By mobilising technological innovation and entrepreneurship to address the consequences of demographic change and strengthen the sustainability of health and care systems, the AAL2 Programme addressed many opportunities of active and healthy ageing and actively contributed to developing a positive perspective on ageing. In doing so, it has opened a space to bring together stakeholder groups that previously had little or no interaction and that worked productively together to develop novel solutions. A wider approach to system transformation is needed to develop the market and overcome the barriers to market impact. The AAL2 Programme evidenced that the demand for digital solutions is shaped by a host of factors, such as cultural expectations towards older age and care, acceptance of technology, the organisation of care delivery and financing within health and care systems. The demand-side is also shaped by a number of intermediaries, such as insurers and public funders. These factors determine whether digital solutions are seen as acceptable and whether they present a likelihood of being taken up. There is an opportunity to tap into the