EUROPEAN COMMISSION Brussels, 15.12.2021 SWD(2021) 383 final COMMISSION STAFF WORKING DOCUMENT Final evaluation of the European statistical programme 2013-2020 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the final evaluation of the implementation of the European statistical programme 2013-2020 {COM(2021) 794 final}



Table of contents 1.Introduction 1.1 Purpose and scope 2.Background to the intervention 2.1 Description of the intervention and its objectives 2.2 Baseline and points of comparison 3.Implementation/State of Play 3.1 Description of the current situation 4.Method 4.1 Short description of methodology 4.2 Limitations and robustness of findings 5.Analysis and answers to the EQs 5.1 Relevance 5.2 Effectiveness 5.3 Efficiency 5.4 Coherence 5.5 EU added value 5.6 Evaluation of the points raised in Article 15 of the ESP 6.Conclusions

Annex 1: Procedural information Annex 2: Stakeholder consultation Annex 3: Methods and analytical models Annex 4: Final Report of the Contractor Glossary Term or acronym Meaning or definition AFS Anti-fraud strategy AAR Annual activity report CMFB Committee on monetary, financial and balance of payments statistics DG Directorate-General EBS European business statistics EC European Commission ECB European Central Bank EEA European Economic Area EFTA European Free Trade Association EQ Evaluation question EQM Evaluation questions matrix ESAC European Statistical Advisory Committee ESCB European System of Central Banks ESGAB European Statistical Governance Advisory Board ESF European Statistical Forum ESP European statistical programme ESS European Statistical System ESSC European Statistical System Committee ESS.VIP European Statistical System vision implementation programme FAO Food and Agriculture Organization FTE Full-time equivalent KPI Key performance indicator IAS Internal Audit Service ISG Interservice steering group JC Judgement criterion MS Member State MAP Multiannual action plan NCB National central bank OECD Organisation for Economic Cooperation and Development OLAF European Anti-Fraud Office ONAs Other national authorities PC Public consultation PMR Planning, monitoring and reporting SDMX Statistical data and metadata exchange SDG Sustainable development goals SMP Single market programme SWD Staff working document UN United Nations UNECE United Nations Economic Commission for Europe UNESCO United Nations Educational, Scientific and Cultural Organisation UNSD United Nations Statistics Division UN-ICSC UN International Civil Service Commission USS User satisfaction survey WHO World Health Organization Definitions Term Meaning or definition Users Users of European statistics include different categories: decision-makers at EU, national and local level; international organisations; academia and researchers; businesses; NGOs; the media; and European citizens in general. Producers Producers or data providers are mostly national statistical institutes (NSIs) and other national authorities (ONAs). Other international organisations producing statistics also provide data in some specific cases.



1.Introduction 1.1 Purpose and scope Article 13 of Regulation (EC) 223/2009 on European statistics states that ‘the European statistical programme shall provide the framework for the development, production and dissemination of European statistics, setting out the main fields and the objectives of the actions envisaged for a period corresponding to that of the multiannual financial framework’. The European statistical programme 2013-2020 was created by Regulation (EU) 99/2013 for the years 2013-2017. It was extended by Regulation (EU) 2017/1951 to cover the remaining period of the previous EU multiannual financial framework, i.e. 2018-2020. Throughout this document, it is referred to as ‘the programme’ or simply ‘the ESP’, while the Regulation creating it is referred to as ‘the amended Regulation’. The general objective of the ESP is for the European Statistical System (ESS) to continue to be the leading provider of high-quality statistics on Europe. As a spending programme, the ESP constitutes the overall framework for developing, producing and disseminating European statistics. The ESP therefore provides the financing for developing and maintaining Eurostat’s statistical infrastructure and the ESS as managed by Eurostat. The ESP also provides financial support to Member States. The requirement to carry out a final evaluation and present a report on the ESP is enshrined in Article 15 of the amended Regulation. Article 15(3) states: ‘By 31 December 2021, the Commission shall, after consulting the ESSC and the European Statistical Advisory Committee, submit a final evaluation report on the implementation of the programme to the European Parliament and to the Council. The report shall in particular evaluate: a)the outcome of the reprioritisation and cost evaluation of statistical products;

b)the actions taken by the ESS to reduce the implementation and production costs for Member States and to limit the overall burden stemming from the statistical projects and fields covered by the programme; c)the progress on rendering access to official statistics easier and more user‑friendly, including the provision of data on the Eurostat website; and d)the progress on the improvement of data availability, including on social economy activities and on the Europe 2020 indicators.’ The evaluation also serves to fulfil the requirements of the Commission’s push for better regulation , and it must therefore follow the better regulation guidelines for evaluations . The evaluation therefore covers the five compulsory evaluation criteria (effectiveness, efficiency, relevance, coherence and EU added value). The conclusions of the evaluation could also be useful when performing the mid-term evaluation of the single market programme (SMP). The SMP has an objective of producing high-quality European statistics and includes the successor of the ESP. Even though the current evaluation focuses on the 2018-2020 period (because 2013-2017 has already been covered by two previous mid-term evaluations of the ESP;), the conclusions of the current evaluation cover the implementation period of the programme in its entirety (2013-2020). The current evaluation encompassed all Member States and other countries covered by the programme. Participation in the programme was open to countries in the European Economic Area (EEA)/European Free Trade Association (EFTA), Switzerland, and countries to which the EU’s enlargement policy applies. Regulation (EU) 2017/951 on the ESP has no related implementing/delegated acts. 2.Background to the intervention 2.1 Description of the intervention and its objectives The general objective of the ESP is for the ESS to continue to be the leading provider of high-quality statistics on Europe. Official European statistics provided under the ESP are indispensable for EU decision-making and for measuring the performance and impact of EU initiatives. High-quality European aggregates, together with comparable statistics on all individual Member States, underpin the investment plan for Europe; the European semester exercise; the European pillar of social rights; the European agenda on migration; and the energy union. Statistics are also key instruments which strengthen the transparency and accountability of EU policies and which enable EU citizens to engage with – and democratically participate in – political life. European statistics serve the needs of a wide range of users, including: (i) decision-makers at EU, national and local level; (ii) international organisations; (iii) academia and researchers; (iv) businesses; (v) NGOs; (vi) the media; and (vii) the European public in general. As a spending programme, the ESP constitutes the overall framework for developing, producing and disseminating European statistics. Its duration was originally planned to be 5 years but this was extended to correspond to the duration of the multiannual financial framework. The ESP provides the financing for developing and maintaining the statistical infrastructure of Eurostat and the ESS as managed by Eurostat. The ESP also provides financial support to Member States. Member States use this support to: (i) improve their national statistical systems; (ii) implement measures to initiate new data collections; and (iii) strengthen the quality and efficiency of statistical production through innovative statistical methods and tools. The statistical infrastructure enables statistics to be regularly produced and disseminated in the policy fields. Depending on the specific needs and uses in these policy fields, the statistical infrastructure also enables new statistics to be developed. The programme’s budget can be roughly split between: (i) developing and maintaining the general methodological and IT infrastructure on the one side; and (ii) providing support to the production, dissemination and development of statistics in the specific policy domains on the other. The total budget allocated for the implementation of the ESP in 2013-2020 was EUR 489 million. The ESP has four specific objectives: ·‘Objective 1: provide statistical information in a timely manner, to support the development, monitoring and evaluation of the policies of the Union properly reflecting priorities, while keeping a balance between economic, social and environmental fields and serving the needs of the wide range of users of European statistics, including other decision-makers, researchers, businesses and European citizens in general, in a cost-effective manner without unnecessary duplication of effort; ·Objective 2: implement new methods of production of European statistics, aiming at efficiency gains and quality improvements; ·Objective 3: strengthen the partnership within the ESS and beyond in order to further enhance its productivity and its leading role in official statistics worldwide; ·Objective 4: ensure that delivery of such statistics is kept consistent throughout the whole duration of the programme, provided that this does not interfere with the priority-setting mechanisms of the ESS.’ The programme is divided into three parts, each focusing on a specific priority area: I. statistical outputs; II. production methods of European statistics;

III. partnership. Objectives 1 and 4 are covered by the measures undertaken in priority area I; objective 2 corresponds to priority area II; and objective 3 to priority area III. The three priority areas are themselves divided into one or two levels of sub-areas. There is a total of 23 second-level, detailed, more operational objectives, spread across the three priority areas. A set of achievement indicators is provided for each of these objectives. Financial contributions in the form of grants to Member States and procurements made by Eurostat are used to achieve the programme’s objectives. Eurostat mainly uses procurement to develop and maintain the general infrastructure, whereas grants help the national statistical systems to develop statistics in specific policy domains. A schematic view of the ESP’s intervention logic is presented in Figure 1 below. The key elements included in the intervention logic are set out in the following bullet points (working from the bottom to the top of Figure 1). ·Needs and challenges: The programme cannot be sufficiently achieved by the Member States alone and can therefore be better achieved at EU level. The ESP addresses a number of needs and challenges observed in the societal, economic and policy spheres. The regulations follow the principle of proportionality and do not go beyond what is necessary to achieve that objective. The annex of the ESP Regulation discusses the needs and challenges faced by the ESS. The diagram in this annex presents a summary version of the needs and challenges encountered by the ESS. Other factors to take into consideration are the evolving nature of complex and multidimensional statistics and the increasing availability of data in the digital age. In addition, the ESP is challenged by the decentralisation of statistics production and the coexistence of multiple statistics producers. Rationalising the production and use of statistics – and improving their efficiency – are key goals to ensure the ESP does not create a disproportionate administrative burden for statistics producers (notably Member States, which are facing significant financial constraints). ·Inputs: EU legislation is an input, a resource provided by the EU. The resources needed to meet the legislative requirements (such as management effort and time spent on the ESS by the Member States, other statistics producers and Eurostat) have also been categorised as inputs. The funding available (through grants and procurements, to: (i) help the national statistical systems develop statistics in specific policy domains; and (ii) develop and maintain the general infrastructure) is also an input. ·Activities: The activities resulting from the implementation of the legal acts (such as the collection, processing and validation of data by Member States, statistics producers and Eurostat) are included in this category. This category also includes activities to communicate and disseminate statistics. Lastly, an ‘overall management’ category includes: (i) the provision of methodological guidance; (ii) the sharing of good practices and standards (including exchange of knowledge between regional, national, European and international levels); (iii) training; (iv) stakeholder dialogues; and (v) participation in international activities. ·Outputs: The outputs of the ESP are the immediate products of the activities implemented. The outputs are also called ‘detailed objectives’ in the Regulation, and they are categorised into three priority areas: statistical outputs, production methods of European statistics, and partnerships. Overall, the three categories have remained stable over time. Figure 1 includes a summary of the outputs/detailed objectives, structured along the three priority areas. Table 1 provides a comprehensive list of the 23 detailed objectives included in the Regulation. ·Outcomes/objectives: In the medium-term, the ESP is expected to achieve four ‘objectives’ (that is the terminology used in the Regulations), also called ‘outcomes’ (Article 4(2) of the 2013 Regulation). These four outcomes are structured along the three priority areas previously identified. The causality within each priority area between outputs/detailed objectives and outcomes/objectives is described in the annex to the Regulation. ·General objective: As per the 2013 Regulation (Article 4.1), the ESP aims to enable the ESS to continue to be the leading provider of high-quality statistics on Europe. ·Strategic objectives: Deciding the strategic objectives goes beyond the ESP, its functioning and governance. The strategic objectives must be chosen to suit the different types of users of the ESP. Through providing high-quality statistics on Europe, the ESP is expected to contribute to: (i) improving policy-making and the measuring of the performance and impact of initiatives; (ii) strengthening the transparency and accountability of policies; and (iii) enabling EU citizens’ civic engagement and democratic participation in political life. The ESP should contribute to these strategic objectives without creating a disproportionate administrative burden – in particular for statistics producers. In Figure 1 , we have differentiated the result levels to indicate when the results are internal to the ESP (outputs, outcomes and impacts) and when they are external to the ESP (strategic objectives). Statistical outputs available to end users Production methods of European statistics Partnership 1. Indicators 1.1. Europe 2020 Objective 1.1.1: provide high-quality statistical information, which should be available in a timely manner for the European Semester, to monitor the implementation of the Europe 2020 targets. 1.2. Economic governance Objective 1.2.1: develop new and improve existing statistical information relevant for EU decision-makers and the public at large on: (i) the strengthened and integrated economic governance of the EU; and (ii) the surveillance cycle integrating the stability and growth pact and economic policy. Objective 1.2.2: provide EU decision makers with reliable statistics and indicators for administrative and regulatory purposes and for monitoring specific EU policy commitments. 1.3. Economic globalisation Objective 1.3.1: improve indicators and statistical information on economic globalisation and global value chains for EU decision makers and the public at large to create a better understanding of globalisation. 2. Accounting frameworks 2.1. Economic and social performance (incl. inequalities) Objective 2.1.1: supplement measurement of economic performance by different aspects of globalisation, quality of life, access to goods and services, environmental sustainability, health, well-being, social cohesion and social inclusion. Develop a framework for analysing globalised production. Objective 2.1.2: provide key macroeconomic and social indicators and principal European economic indicators as a coherent set of indicators addressing EU’s statistical data requirements (and statistical data requirements beyond the EU) and adjust principal European economic indicators (PEEIs) to meet evolving users’ needs. 2.2. Environmental sustainability Objective 2.2.1: provide environmental accounts and climate change-related statistics, taking into account international developments in this area (including environmental footprints). 3. Data 3.1. Business Objective 3.1.1: increase efficiency and effectiveness of statistical production processes, including in light of the better regulation agenda and REFIT. Provide high-quality statistics on key areas where enterprises are the centre of interest. 3.2. People’s Europe Objective 3.2.1: provide statistics on key areas of social policy where the citizen is the centre of interest (with expansion of demographic challenges). 3.3. Geospatial, environmental, agricultural and other sectoral statistics Objective 3.3.1: support evidence-based policy making by a more flexible and increased use of spatial information combined with social, economic, territorial and environmental statistical information. Objective 3.3.2: provide environmental statistics to support EU policy making. Objective 3.3.3: provide energy and transport statistics to support the policies of the EU, in line with the European energy union, and including energy dependency and security of supply. Objective 3.3.4: provide agriculture, fisheries and forestry statistics to develop and monitor the CAP and CFP, reflecting key European strategic objectives in line with the BR agenda and REFIT. 4. ESS quality management Objective 4.1: implement a quality-management system in the ESS based on compliance with the Code of Practice. 5. Priority-setting and simplification Objective 5.1: implement a priority-setting mechanism for the ESS to simplify reporting requirements and to adjust to new needs for statistics. 6. Multi-purpose statistics and efficiency gains in production Objective 6.1: (i) put in place an ESS business architecture allowing more integrated production of European statistics; (ii) harmonise and standardise statistical production methods and metadata; (iii) improve the horizontal and vertical integration of statistical production processes; (iv) use and integrate multiple data sources; (v) produce multi-purpose statistics. Objective 6.2: ensure the good functioning and coherence of the ESS through effective collaboration and communication. 7. Dissemination and communication Objective 7.1: make the ESS the first data source on European statistics for all users; extend and rationalise range of dissemination products, improving user friendliness, retrievability and convertibility. 8. Training, innovation and research Objective 8.1: satisfy learning and development needs in the ESS (including future needs), and ensure the transfer of knowledge and the sharing and implementation of best practices. 9. Partnership within ESS and beyond Objective 9.1: implement the enhanced ESS governance framework. Objective 9.2: improve the coordinating role of the Commission (Eurostat) as the EU’s statistical office. Objective 9.3: strengthen cooperation with the ESCB and European and international organisations involved in the production of data through common projects, ensure consistency between standards. Objective 9.4: promote and implement statistical advisory and statistical assistance activities in countries outside the EU (including awareness-raising activities) in line with the priorities of the EU’s foreign policy (in particular enlargement and ENP). 2.2 Baseline and points of comparison Start of the programme When the ESP’s impact assessment was carried out in 2011, Eurostat found that the ESS faced a number of challenges:

(1) The need for European statistics had been constantly increasing over the preceding years, and it was unlikely that this trend of increased need would change in the future. (2) The nature of statistics was changing – there was a growing need for complex multi‑dimensional statistics of even higher quality which had to be provided within shorter periods of time than before. (3) Due to new participants appearing on the information market providing new types of statistics (including participants providing information almost in real time) the future priority for the ESS would be the freshness of data, in particular for short-term economic information. (4) The situation for statistics producers has become more challenging due to budget constraints both at national and European level. These constraints had become more pertinent with the economic crisis, as had the need to further reduce the burden of compiling statistics on businesses and citizens. The ESP was designed to face all those challenges. For this reason, its main objectives were combining the production of more statistics (including statistics that are more complex and of a higher quality) with the modernisation and reengineering of the processes used to produce these statistics. As a step further in modernising the production of statistics, the ESSC agreed in early 2015 to implement the ESS Vision 2020 , a strategic vision for how the ESS should develop by 2020. The implementation initially involved eight implementation projects for ESS Vision 2020. First mid-term evaluation The ESP’s state of implementation was evaluated for the first time in 2015, taking into consideration the activities carried out and the results obtained in the first 2 years of the programme – 2013 and 2014. The first mid-term evaluation concluded that the implementation of the ESP was advancing well, with 17 of the programme’s 23 detailed objectives well on track to be completed. This first evaluation provided a detailed summary of the ESP’s produced outputs. Good progress had also been made on projects related to modernising the production systems for statistics. The first evaluation stated that the ESP continued to be relevant, as its objectives still corresponded to the EU’s needs and it continued to provide clear EU added value. The evaluation concluded that Eurostat had made efficient use of its resources, both financial and human, with improved productivity at a time when staff numbers were being reduced. The first evaluation said that the ESP was well coordinated with other EU initiatives in the same field, both other Eurostat programmes and initiatives on statistics produced by other Commission Directorate-Generals. The first mid-term evaluation also made the following recommendations: 1.to give special attention to the objectives where problems have been encountered; 2.to try to secure sufficient resources to maintain the necessary level of investment to modernise the production of European statistics; and 3.to identify and implement projects at EU level which could maximise EU added value. Second mid-term evaluation A second mid-term evaluation of the ESP was performed in 2018, covering implementation in 2015-2017. The results of the second mid-term evaluation indicated that the ESP had shown a high degree of effectiveness. The few problems registered in the first mid-term evaluation had been tackled successfully. The percentages of planned outputs that were achieved or on target in the 3 years under examination were constantly over 90%. 20 of the 23 detailed objectives of the ESP could be considered on track to be accomplished by the end of the extended programme. The second mid-term evaluation said that the remaining three objectives ((i) objective 2 on economic governance; (ii) objective 4 on economic globalisation; and (iii) objective 18 on dissemination and communication) only showed limited problems, which would have to be taken care of in the remaining period of the programme. The second mid-term evaluation indicated that the programme: (i) was run efficiently; (ii) provided clear EU‑added value; and (iii) was coherent internally and consistent externally with other initiatives to produce statistics. The results also showed that the ESP had contributed – and continued to contribute – to satisfying user needs and to the design and monitoring of policies. Nevertheless, the results also showed that ESP needed to do more. Eurostat and the ESS were modernising their statistical production processes to respond to user needs while taking advantage of new technologies. The second mid-term evaluation said that the ESP needed to focus on implementing its plan to modernise statistical production processes, saying that this might require: (i) changes in the way statistical authorities gather and process data; and (ii) sufficient resources. The second mid-term evaluation also showed that the ESP’s structure was fairly complex in general, with objectives and sub-objectives covered by areas and measured by indicators which sometimes were not clearly distinct from one another. This made it difficult to understand and monitor the ESP. Therefore, the second mid-term evaluation recommended simplifying the structure of the programme for the future post-2020 ESP. Programme extension 2018-2020 The 2013 programme was extended in 2017 by Regulation (EU) 2017/1951 to cover the period 2018-2020 (the remaining period of the multiannual financial framework up to 2020). The impact assessment accompanying Regulation (EU) 2017/1951 noted limitations in the ESP’s structure relating to: (i) a lack of flexibility to deliver new statistics, while also limiting the associated cost and administrative burden; and (ii) a lack of investment to meet the growing demand for statistics or the need to make statistics available more quickly. The impact assessment recommended an amended programme including: (i) new statistical outputs in line with the 10 political priorities for the Commission’s agenda for jobs, growth, fairness and democratic change; (ii) investment in the statistical infrastructure; and (iii) investment in new sources of statistics (with an increased budget). The extension aimed at: ·providing high-quality statistical information and closing the statistical gaps that needed to be addressed most urgently, focusing on a number of priority areas that reflected the Commission’s 10 political priorities; ·building the permanent capacity needed to respond more quickly to emerging needs and to adapt the statistical infrastructure so as to harness the potential of new data sources; and ·strengthening the partnership within the ESS and beyond to further increase its productivity and secure its leading role in official statistics worldwide.

The amendment of the programme introduced the following three modifications, among others. ·It improved substantially the timeliness of data on inequality, poverty and material deprivation of people in Europe (for instance through flash estimates on the distribution of household income that are fit for purpose for the European Semester). ·It expanded the production of statistics on energy (efficiency, security, renewables, consumption, prices, etc.) to accompany the strategic framework for the energy union and improved the timeliness of these statistics (e.g. issuing flash estimates on energy balances). ·It improved the quality, coverage and timeliness of environmental data to support climate change policy and the circular economy package and to support the UN’s Sustainable Development Goals (SDGs). In the extension of the ESP, the annual budget was increased for the years 2018 to 2020 to a total of around EUR 208 million for the 3 years compared to around EUR 281 million for the previous 5 years. The reason for this increase was to: (i) enable Member States to cope with new demands and to modernise their production systems, as recommended in the first mid-term evaluation of the programme; and (ii) invest in the statistical infrastructure and in new sources of statistics, as recommended in the impact assessment. Programme progress report A progress report on the implementation of the ESP, covering 2018 and the first half of 2019, was produced in 2019. The report indicated that the implementation of the programme was progressing well, producing significant results under the programme’s different objectives. The two mid-term evaluations and the progress report demonstrated that: (i) the delivery mechanism had worked effectively; (ii) the programme was run efficiently; and (iii) the programme was reaching its objectives. They all showed that the ESP provided clear EU added value thanks to the harmonised provision of comparable and high-quality data for the EU. However, the evaluations also showed the need to ensure sufficient resources in the future to modernise the statistical production processes. In addition, the evaluations all argued that Eurostat needed to be able to: (i) respond to users’ growing needs, especially on the timeliness and coverage of new data for emerging policy needs; and (ii) become more agile and take advantage of new technologies. 3.Implementation/State of Play 3.1 Description of the current situation The ESP lasted 8 years, from 2013 to 2020. The total programme budget was EUR 489 million (including EFTA funds and administrative expenditure in direct support of programme implementation). Around 41% was spent on grants and 59% on procurements. Almost another EUR 225 million supplemented the ESP’s own budget (i.e. EUR 225 million in addition to the EUR 489 million budget to make total support of EUR 714 million) in the form of credits sub-delegated by other policy DGs to cover data collections specifically requested by these same DGs. The budget was spent in the programme’s three priority areas. The first priority area, ‘statistical outputs’, deals with the production of European statistics. In this area, the money was mostly spent on grants to provide financial support to Member States. Member States used this support to improve their national statistical systems and to implement measures to carry out new data collections. All Member States benefited from these grants, which ultimately allowed them to produce more and better quality data. As a result, the total number of disseminated datasets increased by 809 – or around 18% – from 2013 to 2020, as detailed in Section 5.3 on efficiency. Source: Eurostat’s dissemination database The second priority area, ‘production methods of European statistics’, deals with improving the way statistics are produced, their quality, and the way they are disseminated. In this area, grants were given to Member States to strengthen the quality and efficiency of statistical production through innovative statistical methods and tools. Member States used this money to participate in the modernisation projects set out in the ESS Vision 2020 programme. In this same area, procurements were used either: (i) to partly finance ESS Vision 2020 projects; or (ii) to finance the modernisation of the statistical infrastructure used to regularly produce and disseminate the statistics. Examples of results under this priority area include: (i) the modernisation of the IT infrastructure to exchange data with Member States; (ii) the renovation of Eurostat’s website; and (iii) the introduction of a series of new visualisation tools and new ways to reach out to users. More details on the ESS Vision 2020 programme can be found in the responses to various evaluation questions (EQs), and in particular under EQ 2 in the chapter on relevance. The third priority area was ‘partnership’. This priority area aimed to support the production and quality of statistics by improving cooperation within the ESS and with other international organisations and countries outside the EU. In this area, the budget was mostly used to support the development – and, to a lesser degree, the production – of statistics in countries outside the EU and EFTA. The budget had a particular emphasis on enlargement and European neighbourhood policy. This priority area enabled Eurostat to regularly publish data for non-EU countries, especially to support the enlargement process and negotiations with those non-EU countries. In spending the budget, there were no major problems and there were no infringements. Eurostat put in place an effective system of anti-fraud measures, which prevented cases of fraud. Different monitoring arrangements were put in place for the ESP. Twice a year, the status of all planned Eurostat activities and their outputs are checked. The percentage of activities and outputs which were on track or achieved has consistently been greater than 90%.

A set of key performance indicators (KPIs) was monitored each year and reported in the programme statements of the ESP, which are part of the annual performance and management reports of the European Commission. The values of the indicators, as extracted from the last programme statement of the ESP, are shown in Table 2 . Source: ESP programme statement for 2020 In 2020, almost all indicators were on track or even surpassing their respective targets. Only the value of the last indicator, related to objective 3, saw a slight decrease in 2020, although it was still performing well. Some of the partnership activities experienced problems in 2020 caused by the COVID-19 pandemic. This was the case for seminars, missions, assessments of statistical systems, and statistics training, all of which were impacted by the restrictions due to the pandemic, especially in countries outside the EU. More details on the indicators can be found under the different EQs, in particular in the chapter on effectiveness. Eurostat wished to continue the activities of the ESP – while adapting them to both: (i) the changing environment, with new possibilities for producing statistics using more sources and innovative technologies; and (ii) changing user needs. To achieve this, Eurostat prepared the statistical annex of the SMP. Drawing on the results of the two mid-term evaluations of the ESP, the 2018 impact assessment on the SMP indicated some important issues to address in the new programme such as: ·the availability of high-quality statistical information to design and monitor new EU policies (e.g. on globalisation, digitalisation of the economy, and security); ·the need for agility to address new demands and provide both faster evidence on emerging topics and deeper analysis of: (i) the effects of globalisation; (ii) new technological developments; and (iii) socioeconomic trends; ·the need to reduce statistical reporting requirements on businesses as well as the public; ·the need: (i) to improve analysis of European statistics on today’s complex realities and interlinkages to better understand them; and (ii) for improved communication and engagement with users, especially EU citizens. The SMP Regulation is applicable from 1 January 2021. The SMP has as a general objective: to develop, produce and disseminate high-quality, comparable, timely and reliable European statistics which underpin the design, monitoring and evaluation of all Union policies and help citizens, policymakers, authorities, businesses, academia and the media to make informed decisions and to actively participate in the democratic process. The specific objective to realise the programme’s general objective is the following: developing, producing, disseminating and communicating high-quality European statistics in line with the quality criteria laid down in Article 12(1) of Regulation (EC) No 223/2009, in a timely, impartial and cost-efficient manner, through a strengthened European Statistical System, referred to in Article 4 of Regulation (EC) No 223/2009, and enhanced partnerships within that system and with all relevant external parties, using multiple data sources, advanced data analytics methods, smart systems and digital technologies, and providing a national and, where possible, regional breakdown. 4.Method 4.1 Short description of methodology The evaluation was performed as indicated in the road map. A contractor, Tetra Tech International Development, was selected via an open call for tenders. Having an external contractor supporting the evaluation process was deemed very important to get an independent opinion on the work carried out by Eurostat. The contractor was charged with:

·designing the evaluation, and in particular the EQs for all criteria and the indicators to respond to them; ·preparing and executing a comprehensive stakeholder consultation; ·collecting and analysing the necessary information; ·compiling the stakeholder-consultation report; ·writing an evaluation report to support the preparation of the Commission evaluation staff working document (SWD). The contract started on 10 September 2020 and the contractor delivered its final report on 23 July 2021. The stakeholder consultation took place between December 2020 and March 2021, and it included a vast spectrum of stakeholders, who were consulted in several ways. The consultation comprised: ·scoping interviews with representatives of Eurostat and other DGs; ·a twelve-week public consultation (PC); ·targeted surveys of users and producers of statistics; ·a large number (50) of interviews of different sorts of stakeholders. The contractor also carried out 4 thematic case studies and 5 country case studies. It collected and analysed many supporting documents. The contractor triangulated information from the different sources to reach robust conclusions. The synopsis report of the stakeholder consultation is included in Annex 2 of the present evaluation. Details of the sources for the contractor’s desk research and the case studies can be found in the company’s final report in Annexes 2, 3 and 4. An interservice steering group (ISG) was set up for the project, including members from all directorates of Eurostat and from many other DGs. The first meeting of the ISG coincided with the kick-off meeting of the contract on 21 September 2020. The ISG then met on 10 November 2020, 6 May 2021 and 6 July 2021. The ISG was able to comment on the different deliverables of the contract, including: (i) the design of the project; (ii) the design of the stakeholder consultation. The ISG was also able to comment on the Commission evaluation report and the SWD. More information on the ISG, including its composition, can be found in Annex 1. 4.2 Limitations and robustness of findings The COVID-19 crisis prevented the contractor’s team from collecting data face-to-face. All activities took place remotely instead. This meant that the team made stronger and more targeted efforts to ensure good response rates for the interviews to be conducted for: (i) the overall programme assessment; (ii) the country case studies; and (iii) the thematic case studies. Overall the level of stakeholder engagement was high. 5.Analysis and answers to the EQs This section presents the analysis and answers to the EQs for the five evaluation criteria. The contractor drew up evaluation sub-questions, judgement criteria (JC) and corresponding indicators to help respondents to reply to the questions, as shown in the evaluation matrix in Annex 3. Using the information collected on the five criteria, the evaluation then also answers the four questions explicitly asked in Article 15 of the ESP Regulation.

A more extensive analysis, that is more detailed level than what is possible in this document, can be found in the final report of the contractor (Annex 4). 5.1 Relevance The ESP provided the framework for developing, producing, and disseminating European statistics, setting out the objectives of the activities envisaged for 2013-2020. It financed the development and maintenance of Eurostat’s statistical infrastructure and of the ESS. The objective of the ESP was therefore to both: (i) fulfil the needs of Eurostat, the NSIs, and other national authorities (ONAs) as producers of European statistics; and (ii) serve the needs of a wide range of users of these statistics. The EQs are presented below. They explored the relation between: (i) the objectives of the programme; (ii) the activities of the programme; (iii) the needs of the ESS; and (iv) the needs of stakeholders more generally. The following sections provide an evidence-based judgement of both: (i) the extent to which the ESP was originally relevant; and (ii) the suitability of the activities to achieve the objectives of the programme. The first EQ focuses on ESS stakeholders, namely producers of statistics, while the second EQ also covers users. Evaluation questions Sub-questions EQ1: To what extent did ESP objectives reflect the needs of the ESS? ·To what extent did ESP objectives correspond to the needs of statistical organisations within the ESS when it was set up in 2013? ·To what extent did ESP objectives continue to meet the needs of statistical organisations within the ESS after its extension and redesign in 2018? ·To what extent did reprioritisation exercises meet the identified needs at the time, especially regarding the 2018 extension? ·What mechanisms are used to ensure the ESP objectives continue to meet statistical organisations’ needs? EQ2: To what extent were ESP activities appropriate to deliver the set objectives of the ESP? ·To what extent were ESP activities appropriate to deliver the objectives set in 2013? ·To what extent do ESP activities continue to be appropriate to deliver the objectives revised in 2018 and to date? ·What mechanisms were used to gather feedback from users? To what extent were these adequate and allowed for adjustment? ·To what extent has the ESP flexibly adapted to technological advances (such as big data)? The replies to the two questions indicate that the ESP was appropriately designed to satisfy stakeholders’ needs. The replies also indicate that, throughout its entire duration, the ESP’s activities were carried out to continue to satisfy these changing needs. At the same time, although the ESP made it possible to implement appropriate activities to meet its objectives, the replies showed that these activities were not enough to deliver all the statistics that users wished for. However, some of the thematic areas that users identified as missing go beyond the scope of the ESP – so non-delivery in these areas cannot be considered as failures in the implementation of the programme. There was some tension between continuously increasing user needs and limitations from producers in keeping up with new demands under an already ambitious programme. This meant that a balance needed to be struck between producer capacity and user requests. In addition, needs related to adaptation to technological advances such as big data became more prominent during the programme, as reflected in the extension of the ESP. These increasingly prominent needs were translated into specific activities, mainly under the ESS Vision 2020 programme. Similar activities will be continued in the next programme, and their results have the potential to provide solutions to some of the problems the ESS still faces in terms of resources and costs. EQ1: To what extent did ESP objectives reflect the needs of the ESS? JC 1.1 ESP objectives and priorities corresponded to the identified needs and gaps of the European statistics sector and to their evolution (i) in 2013; (ii) in 2018 when the ESP was extended and revised; and (iii) at the time of the evaluation in 2020/2021 Stakeholder feedback for this evaluation generally confirmed that the specific objectives of the ESP addressed the needs of the ESS in 2013. Most producers who responded to the targeted survey assessed the four specific objectives of the ESP as relevant to either a great extent or a moderate extent. Most stakeholders interviewed highlighted that the objectives and priorities of the ESP were all relevant and well designed. They also confirmed that the ESP’s specific objectives were aligned with the needs of the ESS and that Member States’ statistical strategies were often based on the ESP’s objectives. The stakeholder consultation did not highlight any mismatch between the objectives and priorities of the ESP and the needs and gaps of the European statistics sector in 2018 when the programme was extended and revised. Producers indicated that the objectives of the ESP had continued to meet their needs. The extension also made it possible to fully align the objectives of the ESP with the 10 priorities of the Juncker Commission, as detailed under JC 2.1. More than half of the producers who responded to the targeted producer survey (55% of 43 overall responses) indicated that they did not see any gaps that the objectives of the ESP did not cover. Nevertheless, a considerable share of the producers who responded (31% of 43 overall responses) did indeed see such gaps. 46% of the 43 that responded also indicated they did not see gaps in terms of meeting future needs, compared to 27% who did. Perceived gaps included the provision of statistics in new or rapidly changing policy areas. The targeted producer survey also revealed limitations in the ESP, including time constraints, challenges in keeping up with Eurostat’s demands, and requests for new statistics. The surveyed producers said these limitations came in addition to the already demanding nature of the ESP. Respondents to the targeted producer survey also felt there was a need to strengthen partnerships in the ESS, and said that Eurostat could coordinate NSIs’ work more closely, as it did during the COVID-19 crisis. Insufficient coverage of access to privately held data was another limitation identified in the targeted producer survey. In addition, respondents to the targeted producer survey and producers interviewed said that an effective system for priority setting (especially for identifying statistics that have become less relevant over time) had been missing until the end of the programme. Eurostat had been conducting an annual priority-setting exercise, in consultation with users and producers of statistics since the start of the ESP in 2013. For the new programme 2021-2027, this priority-setting exercise included a review of statistical requirements in existing domains of European statistics with the intention of reducing costs for Member States. In February 2020, the annual priority-setting exercise was replaced by a new priority-setting mechanism adopted by the ESSC, and a pilot review for climate change related statistics was also put in place. The new mechanism is based on four parts: a multiannual action plan, reviews, user dialogue, and resources/funding. JC 1.2 Mechanisms for feedback and input from ESS stakeholders existed and were perceived as appropriate The ESP relied on a ‘feedback loop’ and input from ESS stakeholders. This feedback loop consisted of the ‘partnership approach’, namely an ongoing discussion between the EU and Member States, which ensured the consideration of different perspectives. As a result, ESS stakeholders consulted by the contractor for this evaluation said that they found there was a good balance in the steering and development of the ESP. Consultation mechanisms were therefore part of the ESP and the ESS stakeholders that were consulted broadly perceived these mechanisms as appropriate. EQ2: To what extent were the ESP’s activities appropriate to deliver the set objectives of the ESP? JC 2.1 The ESP’s activities were the appropriate tools in 2013 to deliver the set objectives, and they continued to be the appropriate tools in both 2018, when the ESP was extended and revised, and again at the time of the evaluation in 2020/2021 80% of the producers who responded to the targeted survey (to which 43 responded) confirmed that the ESP’s activities were appropriate to achieve the four specific objectives – either to a great or to a moderate extent. Given that the objectives of the ESP were formulated at a general level, all NSI activities related to the production of European statistics fell within the scope of the programme. To support the implementation of the ESP, annual work programmes set detailed objectives and outputs for each year. Interviewees mentioned that the annual work programmes made it possible to adapt and adjust the ESP. However, the interviewees also said that the links between the activities of the programme, the ESP’s objectives, and the related indicators were hard to follow, even though Eurostat continuously registered and improved such links. The new ESP multiannual action plan 2021-2027 was designed to tackle this issue. It is intended as a bridge between the SMP, which covers European statistics for 2021-2027, and the annual work programmes. In 2016, the impact assessment for the extension of the programme concluded that there was a need at the time to address several statistical gaps in the ESP’s activities to support EU policies. In addition, the impact assessment said that the relevance of the ESP’s activities was being challenged by: (i) the need for statistics arising from the Juncker Commission’s 10 political priorities; and (ii) the increasing complexity of European society. The impact assessment said that the ESP should therefore be amended and that there also needed to be investment in the statistical infrastructure and new sources of statistics through an increased budget.

In addition, the extension of the programme was an opportunity to: (i) make adaptations and reflect the new policy directions, in line with the ESS Vision 2020; and (ii) complement the ongoing prioritisation in the ESP. To assess whether the ESP’s activities were still the appropriate tools to deliver the set objectives at the time of the evaluation in 2020/2021, the contractor proposed in the evaluation’s methodology to look at their statistical outputs and see whether they served the needs of the wide range of users. On statistical gaps at the time of the evaluation in 2020/2021, users who responded to the targeted survey (to which 45 responded) generally indicated that they would like Eurostat to collect and publish more statistics within its existing statistical themes (64% agreed with this statement while 20% of the respondents were unsure and 16% found the collection and publication of more statistics unnecessary). Respondents highlighted the need for a more detailed and granular breakdown of different types of statistics at regional level. They also highlighted the need for data and statistics subdivided into more detailed sociodemographic attributes, such as gender. Institutional users that were interviewed also identified similar needs that they said the ESP’s activities did not meet. However, producers (of which 43 responded to the survey), were less eager to extend the datasets. 43% of producers indicated that they would not like Eurostat to collect and publish more statistics, while 26% said they would. Producers pointed out the need for Eurostat to keep a balance between: (i) the demand for statistics; (ii) the producers’ (limited) capacity; and (iii) the increased burden new statistics could create for producers. Nevertheless, all 45 users contributing to the targeted survey found that European statistics were appropriate for their purpose, either to a great extent (89%) or to a moderate extent (11%). This indirectly confirms the relevance of the ESP’s activities. The PC (to which 424 responded) also confirmed this, with more than 80% of respondents indicating that European statistics were relevant to their role or sector, either to a great extent (53% agreed) or to a moderate extent (31% agreed). In specific policy areas (assessed through the case studies on business statistics and managed-migration statistics), the evaluation team also confirmed this assessment, as both sets of statistics fitted within the objectives of the ESP and reflected the needs of the ESS. In addition, interviewees from other European Commission DGs confirmed that the ESP’s original activities were appropriate to meet their needs, and said that these activities had continued to be appropriate to meet their needs until that day. Nevertheless, some respondents from the European Commission were of the opinion that the ESP lacked flexibility. They said that any change to the scope of the programme required a change in EU legislation, which is a lengthy process. For example, stakeholder feedback, in the case study on the response to the COVID-19 outbreak, suggested that the outbreak did not activate any specific mechanisms embedded in the ESP to address the challenges the pandemic presented for European statistics. These stakeholders said that the ESP’s activities were neither referred to nor used to shape the response to COVID-19. They said that that the response to COVID-19 was based on informal solutions to unforeseen problems and needs, which formal ESP procedures did not support. However, these stakeholders also said that it was impressive how the ESS was kept running during the pandemic in Eurostat and the Member States, including by shifting to new ways of working. In addition, these stakeholders agreed that Eurostat was agile in leading the work on assessments of how COVID-19 impacted statistics. Also, these stakeholders said that much work took place even if it did not lead to the production of new statistics, such as Eurostat advising colleagues responsible for policy developments on the statistical implications of anti-COVID-19 measures. The stakeholders agreed that this shows that the ESP was set up in a flexible enough way to allow for rapid innovations at a lower level than the ESSC. The continuous and increasing relevance of the ESP is confirmed by the growing number of users accessing European statistics on the Eurostat website. The number of users is estimated via the number of distinct IP addresses that were used to access the database. Source: Eurostat’s website monitoring Another phenomenon that shows the increasing relevance of the statistics produced is the number of data extractions made by users, which increased substantially during the ESP, as detailed under JC 3.2 in the chapter on effectiveness. Yet another indication of the increasing relevance of the statistics produced under the ESP is the continuous growth in the numbers of people following Eurostat on social media. This can be seen in the figures from social-media monitoring, which are presented in Table 3 below. Eurostat opened its Facebook account on 10 January 2017 and its Instagram account on 4 May 2020. Social media 2012 2013 2014 2015 2016 2017 2018 2019 2020 Twitter 10 930 25 080 47 450 73 000 86 775 109 365 127 600 144 000 164 000 Facebook - - - - - 10 400 26 200 42 000 62 000 Instagram - - - - - - - - 9 500 Source: Eurostat’s social-media monitoring Following the demands of the scientific community, Eurostat has increased the numbers of microdata datasets that are available for research purposes. As these datasets contain confidential data, researchers must follow a specific procedure to access them, according to Commission Regulation (EU) No 557/2013 on access to confidential data for scientific purposes. Table 4 below shows the list of datasets and the number of requests for the years 2014-2020. Three new datasets have been published over this period and the number of requests has increased every year until 2019, diminishing only in 2020 due to the COVID-19 pandemic. This is a sign of growing interest in the data contained within these datasets. A single request can be made for more than one dataset. EU-SILC EU-LFS ECHP CIS SES AES EHIS CVTS CSIS MMD ERFT HBS TUS Requests number 2014 164 134 41 32 32 21 13 8 2 N/A 1 N/A N/A 310 2015 206 152 40 37 28 14 15 6 7 4 4 N/A N/A 352 2016 187 131 38 36 28 9 24 6 8 6 9 9 N/A 356 2017 219 173 48 36 50 20 29 14 17 7 5 36 N/A 400 2018 252 179 48 42 48 11 29 6 16 3 3 28 N/A 433 2019 276 183 45 47 46 30 43 15 16 8 9 49 N/A 462 2020 229 183 39 43 44 27 31 18 18 5 5 41 22 404 Source: Eurostat JC 2.2 Mechanisms for feedback and input by users existed and were perceived as appropriate

Bilateral hearings between Eurostat and European Commission policy DGs were organised regularly. These bilateral hearings gave DGs the possibility to give feedback on their needs, including new statistics requests, to Eurostat. They were also an opportunity for the different parties to discuss the overall balance within the programme. The large majority of the requests made by the DGs in these bilateral hearings could be taken into account by Eurostat, which ensured that the ESP’s activities remained relevant for the needs of the main users. These hearings also enabled Eurostat to maintain an inventory of the statistical data produced by other DGs and services, thus helping to avoid possible duplication of statistics. More details on the hearings can be found in JC 10.2 in the coherence chapter. Throughout the period evaluated, Eurostat regularly carried out general user-satisfaction surveys (USSs). These surveys: (i) measured the degree to which Eurostat met its obligations towards its users; and (ii) helped Eurostat ensure that the design of the programme was appropriate, by implementing improvement actions as a follow-up on the feedback received. In addition to the USSs, Eurostat carried out specific usability testing and used feedback gathered through: (i) daily contacts; (ii) questions coming into their user support services; and (iii) comments left on social media. In addition, as part of the ESS Vision 2020, Eurostat engaged in a regular dialogue with users to better understand their needs. Eurostat recognised that different user groups have different needs and planned to address this diversity by offering the right information in the most appropriate way. Eurostat also co-organised the 2014, 2016 and 2018 conferences of European statistics stakeholders to bring together producers and users of statistics to discuss user needs. Even though the documentary analysis showed that mechanisms for feedback and input by users existed, most of the users interviewed were not familiar with these mechanisms and therefore could not assess them. However, this is based on a limited sample of users with different relations to Eurostat, and is therefore not representative. JC 2.3 New technologies and statistical methods have been considered in programme-planning exercises The 43 producers of statistics who responded to the targeted survey confirmed the flexibility of the ESP in adapting to technological advances. Most of these producers indicated that the programme was effective in adapting to: (i) increasing data-security risks (55% said the ESP was effective or very effective in this); (ii) innovation in technology and methods for statistical production (77% agreed the ESP has been effective or very effective in doing so); (iii) statistic variances across countries (79% agreed); (iv) big data (58% agreed); and (v) advances in data visualisation (72% agreed). However, national producers that were interviewed pointed out the lack of a strategy to access and integrate privately held data. The ESS Vision 2020 programme mentioned several challenges the ESS faced in addressing: (i) the new opportunities presented by new technologies and new statistical methods; and (ii) emerging needs (for instance to capture global phenomena and develop statistics with increasing geographical detail). The extension of the programme in 2018 recognised several of these challenges. It also recognised that there were opportunities to improve the timeliness and relevance of official statistics and to reduce response burden. To face these challenges and the continued constraints on resources, the ESS decided to gradually implement the strategic goals set out in the ESS Vision 2020, building upon a holistic approach to deliver quality and efficiency gains. It would implement these goals by: ·using both traditional surveys and other sources, including administrative data, geospatial data and, where possible, big data to deliver statistical products and services; ·getting access to new data sources, creating methods, and finding suitable technology to use such data sources to produce European statistics in a reliable way; ·improving the efficiency of statistical production by further intensifying the sharing of knowledge, experiences, and methodologies, but also by sharing tools, data, services, and resources where appropriate and duly justified; ·implementing a dissemination and communication strategy for European statistics that is flexible enough to adapt to emerging technologies, giving guidance in a world that is undergoing a data revolution, and serving as a reliable pillar of democracy. The ESP made it possible to invest in projects launched in response to new challenges faced by European statistics, including through the ESS Vision 2020. This led to several new projects, including: ·proofs-of-concept for both: (i) the generation of outputs (based on new data sources) in response to user needs; and (ii) addressing horizontal topics related to the use of ‘big data’ for European statistics; ·preparing the IT infrastructure of the ESS so it can exchange large volumes of sensitive data, which in turn supported developments towards the ·secure exchange of intra‑EU trade microdata between all ESS members;, making it possible to both substantially reduce the response burden for businesses and increase quality. Under the ESP, Eurostat developed a variety of data-visualisation tools to better meet user needs. These tools presented data from different statistical themes in a way that was attractive and as understandable as possible so everyone could explore them. Eurostat’s tools used for the dissemination of statistics included: ·visualisation tools, such as the European statistical recovery dashboard, themes in the spotlight, etc. ;

·the My Region mobile app; ·extraction tools, such as the data browser and explorer; ·the ESS experimental statistics hub , which provided a way to disseminate experimental statistics on different topics, including COVID-19 (examples of these experimental statistics include: (i) COVID-19 labour effects across the income distribution specifically; or (ii) quarterly data on registrations and bankruptcies, which can provide critical information on the economic effects of the COVID-19 pandemic ). Mechanisms to adapt to new technologies and statistical methods existed under the ESP in the form of pilot projects for example. This clearly shows that the ESP’s programme-planning exercises considered new technologies and statistical methods. 5.2 Effectiveness The analysis of effectiveness considers how successful the ESP’s activities have been in achieving or progressing towards its objectives. This analysis gives an opinion on the progress made to date and the role of the ESP in delivering the observed changes. It looks for evidence of whether or how these changes are linked to the programme. The EQs, presented below, explore the relation between the objectives and activities of the programme. The answers to these questions provide an evidence-based judgement of: (i) the extent to which the ESP’s objectives were successfully achieved; or (ii) what factors influenced why an activity/objective was (partially) unsuccessful or not yet achieved. Evaluation questions Sub-questions EQ 3: To what extent were the objectives of the ESP 2013‑2020 fulfilled? ·To what extent was the ESP successful in providing timely statistical information (Obj. 1)? Was this delivery consistent (Obj. 4)? ·To what extent was ESP data used in the development, monitoring and evaluation of EU policies? Is there evidence that the ESP contributed to improving policy making (at EU / MS level)? Is there evidence that the ESP contributed to other purposes? ·To what extent did the ESP increase the availability of data, including social economy activities and on the Europe 2020 indicators? ·To what extent was the ESP successful in introducing efficiency gains in the production of European statistics (Obj. 2) and avoiding duplication of effort (Obj. 1)? ·To what extent was the ESP successful in introducing quality improvements in the production of European statistics (Obj. 2)? ·How effective was the ESP in strengthening partnerships within and beyond the ESS (Obj.3)? ·Which factors prevented or reduced the impact of ESP activities? How could these be overcome? EQ 4: To what extent did the ESP make progress on rendering access to official statistics easier and more user‑friendly? ·Were effective feedback mechanisms in place to identify accessibility issues and improvements? ·Did users find the Eurostat dissemination channels (including the website) easy to use? ·Were users provided with sufficient information about key aspects of the data? EQ 5: Did ESP activity result in wider economic, social or environmental impacts? ·Is there evidence that the ESP contributed to wider social, economic or environmental impacts? The evaluation findings show that the ESP was effective in delivering on its objectives, despite remaining weaknesses in the timeliness and completeness of European statistics. Between 2013 and 2020, Eurostat provided high-quality statistics which were used by a wide range of users for a wide range of purposes. These purposes include supporting the development, monitoring and evaluation of policies at the EU and Member-State level. In that period, the ESP also increased the availability of its data and statistics. To a more limited extent, it also introduced efficiency gains in the production of its statistics. In addition, Eurostat effectively developed and strengthened partnerships with its members within the ESS, as well as with partners beyond the ESS – in particular international organisations. However, more needs to be done, in particular to: (i) get access to privately held data; and (ii) create partnerships with private organisations. Work towards both of these goals started in 2020 and is being addressed in the next programme. In addition to achieving its objectives, the ESP was also effective in making access to official statistics easier and more user-friendly between 2013 and 2020. EQ3: To what extent were the objectives of the ESP 2013-2020 fulfilled? JC 3.1 Users expressed satisfaction as to the timeliness and completeness of statistics Objective 1 of the ESP was to provide statistical information in a timely manner. This reflected a particular concern about timely data. This concern was also highlighted in the ESS Vision 2020 programme and repeated in both the 2016 Commission impact assessment on the proposal to extend the ESP, and the 2019 Commission SWD on progress in implementing the ESP. Despite improvement in a number of statistical domains, timeliness remained an area where users said that work was ongoing and needed. The continued issue of timeliness is also reflected in Eurostat’s USSs conducted between 2013 and 2020. In that time, around half of users (results fluctuated between 51% and 54%, and respondent numbers fluctuated between 1 009 and 4 839) rated the timeliness of European statistics as either ‘very good’ or ‘good’ for their purposes. A higher satisfaction rate (63%) was registered in the last USS in 2020 (to which 1 842 responded). Nonetheless, as in previous years, respondents to the 2020 USS continued to ask Eurostat to further improve the quality of statistics by improving timeliness. Similarly, interviews conducted with users and producers of statistics, including stakeholders at the Commission, highlighted timeliness as an issue needing to be addressed. However, interviewees also acknowledged that quality official statistics required more time to be produced, and confirmed that there was therefore a trade-off between timeliness and other aspects of quality. Lastly, the timeliness of statistics also fluctuated between domains/statistical themes. However, while timeliness was identified as a challenge, improvements in that regard were also highlighted by users. For instance, 51% of respondents to the PC (to which 424 responded) agreed that ‘the ESP had been successful in improving the production of European statistics in regard to timeliness and punctuality’ to a great extent, while 28% said this was true to a moderate extent, and only 5% said it was true to a lesser extent. Similarly, 82% of respondents to the targeted user survey (to which there were 45 replies) thought that the ESP was successful in improving the timeliness and punctuality of European statistics. Progress reported against the ESP objectives confirmed these perceptions. Indeed, the percentage of outputs achieved or on target for strategic objective 1 has consistently increased since 2016, reaching a peak in 2019, as detailed under JC 3.6 (see Figure 5 ). On the completeness of statistics, based on the USS, half of users between 2013 and 2019 (with results fluctuating between 49% and 52%, and respondent numbers fluctuating between 1 009 and 4 839) rated the overall completeness of European statistics as either ‘very good’ or ‘good’ for their purposes, with a peak at 62% in 2020. The issue of completeness was also raised during interviews with users who highlighted gaps, particularly when data for certain countries were not available. These users also noted inconsistencies in how data was collected for different countries. Respondents to the targeted producer survey judged the availability of statistical data as sufficient, pointing out limitations due to confidentiality. In contrast, the producers of statistics that were interviewed assessed the availability of data as very good, with the right balance between timeliness and availability of data. Similarly, users at the European Commission stressed that the statistics provided under the ESP were comprehensive, comparable, and reliable. JC 3.2 European statistics and data were downloaded and used by a range of stakeholders for a range of purposes Serving the needs of a wide range of users remained a specific objective (objective 1) of the programme throughout the period under evaluation. According to the 2020 USS (to which 1 842 responded), 39% of respondents identified themselves as advanced users, 35% as intermediate users and 26% as light users. Respondents belonged to all categories, from ‘students, academics and private users’ to ‘business’, ‘government’ and ‘EU and international organisations’. This suggests that European statistics were indeed downloaded and used by a range of stakeholders.

Web analytics confirmed that European statistics are downloaded at an increasing rate. Between 2014 and 2020, the number of data extractions made by external users from Eurostat reference databases greatly increased from 6 813 000 extractions to 21 480 000 (+315%) as shown in Figure 4 below. Similarly, between 2014 and 2020, the number of distinct IP addresses consulting European statistics increased from 3 441 157 visits to 11 731 274 (+240%). The number of requests submitted for microdata also increased until 2019, while diminishing in 2020 due to the COVID-19 pandemic. Source: Eurostat’s website monitoring Users of European statistics were asked in the 2019 USS (to which 1 009 responded) to indicate the reason for their interest in using European statistics. Among the responses available, ‘monitoring or formulating policy’ (chosen by 32%) and ‘general background information’ (chosen by 23%) were the most commonly reported purposes. Looking at the importance of European statistics, more than three quarters of participants (77%) indicated them to be either ‘essential’ or ‘important’ for their work. Building on the USS, a targeted user survey (to which there were 45 responses) was carried out, in which respondents were asked to indicate the usefulness of European statistics for nine areas of practice. Six of these areas (development, monitoring and evaluation of policies at the EU level; development, monitoring and evaluation of policies at the Member-State level; academic research; scientific research; media reporting; and giving information on Europe) received a positive assessment (more than 60% said European statistics in these areas were either ‘useful’ or ‘very useful’). JC 3.3 Eurostat data were used in journalistic and research activity European statistics were used in journalistic and research activities, even if these activities were not the main use of these statistics. According to the targeted user survey (to which there were 45 responses), 9% of respondents indicated that they used European statistics for ‘academic/research’ purposes. Similarly, 12% of respondents to the 2019 USS (to which there were 1 009 respondents) said they used European statistics for ‘research’ purposes, which was the fourth most commonly selected main purpose, while 1% indicated that they used the statistics for ‘media use’. As counted by the Commission’s ‘My news’ tool, Eurostat was mentioned in the media more than 6 000 times per month on average in the last months of 2020. JC 3.4 Statistics were used by EU policy makers to inform policy making and decisions The reformulation of objective 1 in the programme extension in 2017 stressed that decision-makers at EU level (but also in Member States, local governments and business) needed European statistics ‘to prepare, apply, monitor and assess all EU policies’. The findings from the USS, the targeted user survey, and the PC suggested that European statistics were indeed used by policy makers to inform policy making and decisions. In the 2019 USS (to which there were 1 009 responses), the most common purpose identified by users of European statistics was ‘monitoring or formulating policy’ (selected by 32%). Additionally, ‘preparing legislation’ and ‘monitoring or formulating policy’ both got a combined share of 87% of participants who indicated them to be ‘essential’ or ‘important’ for their work. Similarly, 64% of respondents to the PC (to which there were 424 responses) considered European statistics useful to a great extent in informing and monitoring policy and decision-making in Europe. Interviews with users from the Commission confirmed these findings. Commission users systematically mentioned the comprehensiveness, comparability and reliability of the data provided under the ESP and the value of this data for EU policy work. For instance, the European statistical recovery dashboard for tracking the recovery from COVID-19 was highlighted by policy makers in several DGs as important to ‘inform and support policy makers’ and to ‘respond quickly to emerging needs’. In the case studies, regional data were also reported as being heavily used for policy planning, especially in the policy area of regional development. Similarly, migration statistics were reported by DG Migration and Home Affairs as important for informing EU migration policy, policy briefs, and legislative proposals. JC 3.5 Data were being used by Member State policy makers to inform policy making and decisions The findings outlined in this JC suggested that European statistics were also used by policy makers at Member-State level, albeit: (i) to a lesser extent than by EU policy makers; and (ii) to a greater extent at the national level than at the regional and local level. Further breakdown of the data by contributors showed that among the respondents to the PC (to which there were 424 responses), 81% of respondents from public authorities with a national scope of work considered European statistics relevant to inform and monitor public policy. When asked about the usefulness of European statistics in the targeted user survey (to which there were 45 responses), 30% of respondents found them to be ‘very effective’ and 52% ‘effective’ in the ‘development, monitoring and evaluation of policies at the Member State level’. European statistics were found to be less effective at the regional level (where only 31% found them effective) and at the local level (where only 10% found them effective). This could be linked to the insufficient disaggregation of statistics at local level. JC 3.6 ESP activities resulted in a net increase of available data; users considered newly available data sources to be useful When asked in the targeted producer survey (to which there were 43 responses) about the availability of European statistics, more than three quarters of respondents indicated there had been an increase in the availability of data to a great extent (37% agreed) or to a moderate extent (44% agreed). Similarly, in the targeted user survey (to which there were 45 responses), respondents found that there had been an increase in quantity for general and regional statistics (44% agreed) and for economy and finance (45% agreed). A noticeable share (43%) of respondents to the PC (to which there were 424 responses) also reported an increase in the availability of data on population and social conditions. This was further corroborated by stakeholders interviewed as part of the asylum and managed-migration case study. Interviews conducted with users of European statistics, producers of European statistics, and people from EFTA and candidate countries confirmed this point. Table 5 below shows the quantity of statistics and long-time series published by Eurostat. Statistical coverage is calculated as the number of statistical indicators, sub-indicators and all their breakdowns included in Eurobase (the Eurostat dissemination database) not taking into account those differing only because of the time dimension. The table thus demonstrates the increasing amount of data made available by Eurostat over the years. Year 2017 2018 2019 2020 Statistical coverage 328 366 423 446 Long time-series 35 42 50 55

Source: Eurostat reference database The ESP’s great effectiveness in producing statistics is further demonstrated by the percentage of planned outputs which were achieved under ESP objectives 1 and 4, which were constantly above 92% (and above 95% in the second half of the programme), with only 3% or less of outputs cancelled or not fully achieved. Source: Data from Eurostat Annual Activity Reports 2014, 2015, 2016, 2017, 2018, 2019 and 2020. (*) for the year 2014 and 2015, objective 1 and 4 are combined Furthermore, the few limited problems encountered in three detailed objectives in the second mid-term evaluation of the programme were all solved by 2019 The actions implemented to solve these problems included: (i) a pilot to test the remote-access system connecting safe centres in national statistical institutes (NSIs) to the central node; (ii) an SWD covering progress on the European public-sector accounting-standards project; and (iii) carrying out a survey on international sourcing. JC 3.7 ESS stakeholders and/or users considered that ESP activity increased the efficiency of European statistics production Specific objective 2 of the ESP was to implement ‘new methods of production of European statistics aiming at efficiency gains’. As this objective relates to efficiency, a detailed answer to this JC has been provided under the chapter on efficiency. Overall, both the qualitative and quantitative evidence collected suggest that ESP activities have, to a limited extent, resulted in efficiency gains by increasing the use of new, more efficient methods of production of European statistics while reducing administrative burden. However, although the consulted stakeholders acknowledged efficiency gains implemented by the ESP, they also said that more innovative solutions and sources should be used to further increase efficiency. Eurostat launched a set of projects to increase the efficiency and quality of the process for producing European statistics. These projects included: (i) implementing the ESS Vision 2020 programme; (ii) strengthening the European statistical infrastructure; and (iii) making better use of big data and shared infrastructure, as detailed under JC 2.3 in the chapter on relevance. JC 3.8 ESS stakeholders considered that ESP activity had avoided or prevented duplication of effort on the part of stakeholders As laid down in its 2016-2020 strategic plan, Eurostat sought to provide high-quality statistics in a ‘cost effective manner without unnecessary duplication of effort’. This was achieved through: (i) sharing knowledge and best practices across Member States; and (ii) developing new technologies, common tools and collaborative networks in the ESS for ‘taking advantage of possible synergies and avoiding duplication of effort’. Eurostat therefore produces European statistics in partnership with the Member States based on shared statistical standards, methods, procedures, practices, and tools. Respondents to the targeted producer survey (to which there were 43 responses) indicated that to a moderate extent (51% agreed) and to a great extent (19% agreed) the ESP had been successful in reducing duplication of effort. Additionally, stakeholders within the Commission highlighted the ongoing dialogue between Eurostat and policy DGs on the ‘inventory of other statistics’ as being effective at preventing overlap and duplication of efforts. Eurostat’s interviewees also highlighted the European Commission’s participation in discussions in international partnerships. Some of these discussions focused on efforts to limit the response burden for Member States that are statistics producers for different international organisations by avoiding the duplication of statistics production. JC 3.9 ESS stakeholders and/or users considered that ESP activity had increased the quality of European statistics production As mentioned above, specific objective 2 of the ESP was to implement ‘new methods of production of European statistics aiming at ... quality improvements’. The indicator on timeliness already covered this objective. Additionally, the percentage of users that rated as ‘very good’ or ‘good’ the overall quality of European statistics remained constant between 2013 and 2019 (fluctuating between 57% and 60%, with respondent numbers fluctuating between 1 009 and 4 839) and increased considerably in 2020 (when there were 1 842 responses to the USS) to 72%. The increase in 2020 will have to be checked in future surveys to confirm if the overall quality of statistics has improved. Nevertheless, it should be noted that overall quality is rated more favourably than individual quality dimensions. Similar findings were reported in the targeted user survey (to which there were 45 responses) where respondents were asked to judge the overall quality of nine of Eurostat’s statistical themes. Overall, respondents said that all themes were of either high or moderate quality. However, when asked whether there had been changes to data that improved the quality of the statistics available, for most themes, respondents were unsure if there had been such changes. The only themes in which changes were noted were: (i) economy and finance where 58% of respondents reported having noticed changes to the data that improved the quality of the statistics available; and (ii) population and social conditions where 39% of respondents noted changes. Producers of statistics, users of statistics, European Commission DGs, EU bodies, and stakeholders from EFTA and candidate countries all emphasised in their interview the high quality of the statistics produced by Eurostat. Additionally, and in contrast to the surveys’ findings, interviewees were positive about the progress which had been made in improving the quality of European statistics production. Overall, the quality was assessed by all stakeholders as being high, which would mean that there was little room for improvement and would therefore suggest that the small increase witnessed still represents an improvement. JC 3.10 ESP activity strengthened existing partnerships or developed new ones The ESS is a partnership between Eurostat and the NSIs and ONAs responsible in each Member State for developing, producing and disseminating European statistics. This partnership also includes the EEA and EFTA countries. Specific objective 3 of the ESP was to ‘strengthen the partnership within the ESS and beyond in order to further enhance its productivity and its leading role in official statistics worldwide’.

Progress made against this objective has been reported on in Eurostat’s annual activity reports (AARs). An analysis of the AARs in recent years shows that the percentage of outputs achieved or on target for strategic objective 3 has been high since 2014, suggesting that ESP activities have been successful in strengthening existing partnerships. Source: Data from Eurostat AARs 2014, 2015, 2016, 2017, 2018, 2019 and 2020. Additionally, the comparability of European statistics was one of the important expected results of the partnership within the ESS. According to the USSs, the percentage of respondents viewing the comparability of European statistics as either ‘very good’ or ‘good’ increased between 2014 (when it 50% assessed comparability as either ‘very good’ or ‘good’, from a survey to which 4 839 replied) and 2019 (when it was 53%, from a survey to which 1 009 replied) with a jump in 2020 (when 1 842 responded to the survey) to 58%. This was confirmed in the interviews with users, who highlighted the close and intense cooperation with Eurostat. Producers of statistics in the Member States reported an increase in – and strengthening of – partnerships. They also reported a good overall level of cooperation across the ESS. On the COVID-19 crisis, these producers of statistics said that meetings and exchanges to share good practices and new methods had increased since the outbreak of the pandemic. Examples of aspects that worked well in the partnership and which were highlighted by interviewees were: (i) the working groups, taskforces and meetings between Eurostat’s Director and the directors of the NSIs; and (ii) the annual hearings organised bilaterally between Eurostat and the policy DGs. In interviews, EFTA and candidate-country stakeholders also noted the strengthening of partnerships taking place within the ESS. The evaluation found that Eurostat cooperated closely with international organisations to: (i) align their methodologies; (ii) agree on definitions for individual statistical indicators; and (iii) validate and exchange data. Eurostat currently has memoranda of understanding (MoUs) signed with four organisations: (i) the Organisation for Economic Cooperation and Development (OECD); (ii) the ECB; (iii) the UN International Civil Service Commission (UN ICSC); and (iv) the International Service for Remunerations and Pensions (ISRP). Eurostat has also signed administrative arrangements with four organisations: (i) EFTA; (ii) the UN Statistics Division (UNSD); (iii) the Food and Agriculture Organisation (FAO); and (iv) the OECD. Stakeholders interviewed from these organisations were overwhelmingly positive about their partnerships with Eurostat, highlighting a close and fruitful collaboration. Additionally, according to the 2019 final audit report on the effectiveness of Eurostat’s cooperation with external stakeholders, this cooperation has had ‘a significant impact on the implementation of the ESP’. However, despite an overall positive assessment, this final audit report also highlighted some challenges. To address these challenges, corrective actions were undertaken in 2019 and 2020, including: (i) the development of a strategy for international cooperation; (ii) the revision of internal guidance on administrative arrangements; and (iii) the preparation of formal agreements to be signed with external stakeholders. The Internal Audit Service (IAS) concluded in its follow-up report in 2021 that the recommendations had been adequately and effectively implemented. Eurostat also cooperates with non-EU countries based on regional and thematic priorities to exchange information and practices that help to address global challenges. As part of its work to create a quality framework for big data, as first stipulated by ESGAB Recommendation 2017/4, Eurostat has, as part of the ESS Vision 2020 BIGD project, worked on developing a methodology and practices for dealing with privately held data and their quality aspects. However, Eurostat considered that access to privately held data was still insufficiently covered and more needed to be done in this area. In its 2020-2024 strategic plan, Eurostat is planning a stronger partnership with the private sector to improve data-production processes. JC 3.11 Stakeholders identified factors that prevented or reduced the impact of the ESP’s activity Producers of statistics stressed the need for more innovative solutions and sources to be used, such as big data based on privately held data. However, using such data requires specific skills traditionally not available in statistical offices. It also poses complex problems, such as: ·the lack of incentives and enablers, including in harmonised legal provisions, for private data holders to contribute to the production of official statistics for public benefit; ·the perceived risks for compliance with data-protection legislation and reputational risks linked to privacy protection on the part of private data holders; ·the high initial investment needed to re-purpose data and to develop methodological and quality frameworks related to the use of new data sources. Eurostat is aware of these problems and will work together with the ESS to solve them. Work is already planned under the SMP and included in the 2021-2027 multiannual action plan. Problems with timeliness reduced the impact and use of the statistics being produced by Eurostat. However, respondents and interviewees acknowledged that timely statistics were often dependent on external factors, namely Member States’ capacity to provide statistics in a timely manner. Interviewees also acknowledged that quality official statistics needed time to be produced and verified. Interviewees also agreed that an additional challenge to timeliness came from the fact that the EU’s legislative cycle is relatively long. Stakeholders from Eurostat said that modernisation required an upfront investment which Member States may be reluctant to pay. They said that this had impacted the ESP’s ability to deliver efficiency gains to reduce burden in the medium-term. The stakeholders said that the impact of COVID-19 was likely to produce financial restraints on the EU and on Member State budgets dedicated to producing statistics. They said that, as a result, there was an increased risk of delays in achieving the objectives of modernising statistics production. An additional factor that limited the impact of the ESP was the increasing risk of third-party dissemination of non-validated or fake information. In response, the ESGAB recommended in its 2019 report that Eurostat and the NSIs design appropriate actions of communication and outreach to highlight the trustworthiness of official statistics. To address this problem, ESAC also recommended promoting statistical literacy at all educational levels and improving the public’s statistical literacy to promote the value of official statistics. EQ4: To what extent did the ESP make progress on rendering access to official statistics easier and more user-friendly? JC 4.1 Effective processes were in place to monitor and receive feedback on data accessibility

User feedback is the best way to assess the accessibility and clarity of European statistics. To monitor feedback from its users, Eurostat carried out several general USSs throughout the period under evaluation. The objective of these surveys was to measure the degree to which Eurostat met its obligation towards its users. The surveys were designed to obtain better knowledge about users, their needs and their level of satisfaction with the services and statistics provided by Eurostat, including in the area of data accessibility. USSs were held in 2013, 2014, 2015, 2016, 2017, 2019 and 2020. In addition, input and feedback were also received through daily contacts from: (i) questions coming into the user-support service; (ii) comments left on social media; and (iii) a staff feedback mechanism. These forms of feedback fed into the revision and creation of new and existing communication materials. When asked in the PC (to which there were 424 responses) about Eurostat’s communication channels, more than half of the respondents (53%) said they found that these channels had been effective to a great extent in responding to their feedback and input, while 35% found them effective to a moderate extent. The same question was asked in the targeted user survey with similar results. Both users and producers of statistics were satisfied with the processes in place to guarantee effectiveness. However, many users may not be familiar with the feedback mechanisms, as indicated in the interviews. JC 4.2 Relevant information (e.g. new releases, data-quality standards) was available in a clear and accessible manner The main way to access European statistics was through the Eurostat website, which has provided users with free access to its databases and all its communication and electronic publications. The Eurostat website was updated daily throughout the period of the ESP, and gave visitors access to the latest and most comprehensive statistical information available on the EU, EU Member States, EFTA countries, and candidate countries. The Eurostat website also provided access to a narrower range of statistics covering non-members of the EU, including potential candidate countries, neighbouring countries, and other major economies, such as Japan and the United States. People who registered (free of charge) were able to receive tailor-made email alerts providing information on new publications as soon as they were online. They also had access to enhanced functionalities within databases (for example, the ability to save data queries and make bulk downloads). Journalists were able to obtain, on the day of publication, news releases or a weekly release calendar by email in German, English, or French. Data were made available for download from the website’s data explorer in various formats (XLS, CSV, HTML, PC AXIS, SPSS, TSV and PDF). Similarly, Eurostat’s dissemination unit also created different profiles for users to ensure that they receive tailored communication packages. Eurostat’s website was completely redeveloped at the beginning of the programme. The new website was launched in December 2014 and it has been continuously improved ever since. Since the beginning of the ESP, Eurostat has developed and added many new: (i) electronic and interactive publications; (ii) visualisation tools; (iii) mobile apps; and (iv) tools for data extraction. These tools present data from different statistical themes in an attractive and easy-to-understand way for everyone to explore and can be easily accessed on Eurostat’s website . More details can be found under JC 2.3 in the chapter on relevance. Adding modern ways of communication, Eurostat has also been active on social media, with its three corporate social-media accounts on Twitter , Facebook and Instagram . Users that were interviewed provided positive feedback on Eurostat’s publications, with the publication ‘Statistics explained’ being referred to several times as useful and well written. Different aspects of the publications (such as detailed metadata with descriptions, sources and methods) were also deemed useful and easily accessible via links. However, some limitations in accessibility and clarity were revealed in the USS. On average, in 2019 (when there were 1 009 responses to the USS), only 45% of respondents found metadata sufficient for their purposes (48% found them partially sufficient, and 7% not sufficient). Additionally, although some users reaffirmed in their comments that metadata were clear, complete, and better than those of other data providers, others found the metadata difficult to access, not clear enough, too long, or too technical. The main improvements suggested by users included: (i) providing some more basic metadata that was easy to understand and written in plain language for non-specialists; (ii) providing metadata at a more detailed level and for all indicators; and (iii) giving clearer and more complete definitions of all codes. Other respondents wished to get more information on the production of statistics and the methodology used, and to understand more easily the differences between countries. They also said that metadata should be consistent over time and among different statistics, and always updated if there were changes in the methodology. In both the targeted producer surveys and the targeted user surveys, respondents were asked to rate the extent to which Eurostat was providing sufficient information about the sources of the data, the data-quality standards, and the methods of data collection. Answers were mostly positive in all cases, suggesting that both producers and users found relevant information on the data produced by Eurostat to be available in a clear and accessible manner, with producers being slightly more positive in their feedback than users. Overall, and despite the weaknesses identified (notably with metadata), the findings suggested that relevant information was provided by Eurostat in a clear and accessible manner. EQ5: Did ESP activity result in wider economic, social, or environmental impacts? JC 5.1 Examples of wider social, economic, or environmental impact were identified by stakeholders and attributed in part or in full to ESP activity Identifying the direct social, economic, or environmental impact of statistics is difficult. This is because statistics ordinarily only have an indirect impact through the policies that are based on them. However, stakeholders at Eurostat highlighted that the ESP had had a wider direct social impact, thanks to training, skills, and the diffusion of innovation. One example of this is the creation of the European Master’s in Official Statistics (EMOS) to strengthen collaboration among academics and producers of official statistics, and to help develop a professional network of statisticians. Similarly, interviewees mentioned the development of the ‘trusted smart statistics’ hub on web intelligence. At this hub, all NSIs are provided with the opportunity to explore the web as a data source, and to use algorithms and coding for free to illustrate the wider social impact of Eurostat activities. Additionally, Eurostat has an ‘ education corner ’ page on its website where teachers can find material to use in the classroom when teaching statistics, geography, social science, etc. The education corn