EUROPEAN COMMISSION Brussels, 20.1.2023 SWD(2023) 13 final COMMISSION STAFF WORKING DOCUMENT



EVALUATION

Accompanying the document Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on European statistics on population and housing, amending Regulation (EC) No 862/2007 and repealing Regulations (EC) No 763/2008 and (EU) No 1260/2013







{COM(2023) 31 final} - {SEC(2023) 38 final} - {SWD(2023) 11 final} - {SWD(2023) 12 final} - {SWD(2023) 14 final} - {SWD(2023) 15 final}



Table of contents 1.Introduction4 1.1. Political and legal context 4 1.2. Purpose of the evaluation 5 1.3. Scope of the evaluation 6 2.Background to the intervention6 2.1. Initial problem/needs statement 6 2.2. Description of the intervention and its objectives 7 2.3. Baseline and points of comparison, including evolution of policy needs 9 3.Implementation/state of play13 3.1. Inputs and external factors 13 3.2. Description of the current situation 14 4.Method16 4.1. Short description of methodology 16 4.2. Limitations and robustness of findings 18 5.Analysis and answers to the evaluation questions19 5.1. Relevance – initial situation (RI) 19 5.2. Relevance – evolution until today (RE) 21

5.3. Effectiveness (EE) 23 5.4. Efficiency (EI) 27 5.5. Coherence – internal (CI) 34 5.6. Coherence – external (CE) 36 5.7. EU added value (EU) 38 5.8. Statistical quality (SQ) 41 6.Conclusions46 6.1. What is working well 46 6.2. What is not working well 46 6.3. Lessons learnt 48 Annex 1: Procedural information49 Annex 2: Methods and analytical models56 Annex 3: statistical datasets of European statistics on population published under the intervention68 Annex 4: Complete evaluation framework90 Glossary Term or acronym Meaning or definition Census Census Hub Web tool for central access to European census outputs Census Regulation Regulation (EC) No 763/2008 on population and housing censuses CES Conference of European Statisticians Demography Regulation Regulation (EU) No 1260/2013 on European demographic statistics ECOFIN Council Economic and Financial Affairs Council EEA European Economic Area EFTA European Free Trade Association EPC Economic Policy Committee ESOP European statistics on population ESS European Statistical System ESSC European Statistical System Committee EU European Union Eurobase Public database of European statistics disseminated by Eurostat Eurostat Statistical office of the European Union FTE Full-time equivalent ISG Interservice group of the European Commission Migration Regulation Regulation (EC) No 862/2007 on Community statistics on migration and international protection MS Member State(s) of the European Union NSI National statistical institute OPC Open public consultation TEC Treaty establishing the European Community TEU Treaty on European Union TFEU Treaty on the Functioning of the European Union UNECE United Nations Economic Commission for Europe 1.Introduction 1.1.Political and legal context

According to Article 9 of the Treaty on European Union (TEU), every national of a Member State, in addition to their national citizenship, is also a citizen of the European Union (EU). To develop policies to benefit the people of Europe, EU institutions need timely, reliable, detailed, harmonised and comparable European statistics. EU institutions also need a reliable and comparable count of the whole population of the EU. This will help the institutions to uphold the principle of non-discrimination in all their activities, and to defend individual citizens’ rights as enshrined in Article 10 of the Treaty on the Functioning of the European Union (TFEU) and the Charter of Fundamental Rights of the European Union. The Commission is required to monitor and report on the EU’s demographic situation in line with Article 159 TFEU. EU institutions also need accurate and comparable population figures for administrative and procedural purposes, e.g. for qualified majority voting in the Council. Population statistics are the backbone of all social statistics, as they provide the most accurate and up-to-date reference information on the entire population and its basic demographic characteristics. An accurate picture of the population, with very good coverage and location information, is indispensable for any more detailed annual population estimates, sample surveys, and regional analysis. Population estimates are also needed to obtain per capita indicators in statistics. Population statistics provide the input for preparing population projections for the EU’s long-term economic and budgetary projections in particular. Population statistics are also useful more generally for formulating and implementing the EU’s economic, social and cohesion policies. The Treaties oblige the European Parliament and the Council to adopt measures for producing official statistics where necessary for EU policies (Article 338 TFEU, formerly Article 285 TEC). Over the past three decades, many EU policy areas have experienced strongly increasing and evolving needs for complete, coherent, comparable, reliable and regular European statistics. The statistics needed in these policy areas cover population, demography and international migration, and are crucial to support evidence-based policymaking. After an initial period of voluntary collections of data from Member States, various EU institutions expressed a need for a better common basis for population and migration statistics, including for legislation at EU level. Therefore, the Commission (Eurostat) initiated legislative work in 2005 on several legal instruments to establish a legal basis for these statistics to address policy needs in a proportionate way. Relevant legal acts adopted as part of this legislative work include: ·Regulation (EC) No 763/2008 on population and housing censuses (‘Census Regulation’); ·Regulation (EU) No 1260/2013 on European demographic statistics (‘Demography Regulation’); ·Regulation (EC) No 862/2007 on Community statistics on migration and international protection (‘Migration Regulation’), in particular Article 3 thereof on international migration, migrant stocks and acquisitions of citizenship. The Commission also adopted various implementing measures to help implement these regulations. In this staff working document, we will use the term ‘the intervention’ as a general term to describe these three Regulations and their implementing measures. Many EU policy areas have become increasingly dynamic over the past decade in response to social, economic and environmental developments. These developments include demographic changes, migration and the increasing exposure of Europeans to natural disasters in the wake of climate change. Even while this intervention was being implemented, evidence-based policymaking in many areas has continued to evolve. This policymaking now requires even more harmonised, detailed, frequent and timely European statistics on population and migration. Many of these policy areas also require statistics on small or functional geographies (e.g. grids and cities, and breakdowns by functional area such as urban and rural areas). Moreover, statistical needs are expected to continue to change rapidly in this increasingly dynamic policy and societal environment. 1.2.Purpose of the evaluation The purpose of this evaluation is to assess official statistics at EU level on population, demographic events and international migration. The evaluation aims to assess whether these statistics have been providing sufficient data evidence to support both EU policymaking and the functioning of EU political decision-making. This includes evaluating the relevance, coherence, effectiveness, efficiency, EU value added, and statistical quality (a contextual ad hoc criterion) of all mandatory and voluntary European statistics produced in these domains. In particular, relevance will be measured against: (i) the initial policy and institutional needs before 2005(at the start of the intervention), and (ii) current and evolving needs over time. The geographic coverage of the statistics in question is the European Economic Area (EEA), and the time coverage includes all reference dates and periods of statistics between 1 January 2008 (the first reference year of the Migration Regulation) and 31 December 2020. This means in particular that census outputs only from a single round (2011) can be fully evaluated. This is because statistical outputs from the previous 2001 round were produced before work began on the intervention, and outputs from the ongoing 2021 round will not be available for the whole EU at the time of concluding this evaluation. However, administrative and procedural aspects of the 2021 EU census round (e.g. cost and burden estimates, coherence with international recommendations) will be included in this evaluation. The results of this evaluation will inform a decision on whether follow-up action is needed to update or redevelop the existing legal base given current and further evolving needs for policies and decision-making in the EU. 1.3.Scope of the evaluation This evaluation covers European statistics that describe the population residing in the EU. The key underlying concept is thus the place of residence of a person according to a given definition. Currently, annual statistics on demographic characteristics of the resident population are published under the Demography Regulation. This evaluation also covers more detailed statistics in areas such as the family, household and housing situations of the resident population. These more detailed statistics are currently published every 10 years under the Census Regulation. Finally, this evaluation will also consider annual statistics published under Article 3 of the Migration Regulation on: (i) migrant stocks (resident population with migration history); (ii) international migration flows (changes of residence between countries both within and entering/leaving the EU); and (iii) citizenship. A comprehensive table of datasets used for this evaluation is provided in Annex 3. A number of statistical collections are out of the scope of this evaluation. For example, statistics on asylum and managed migration under Articles 4 to 7 of the Migration Regulation address administrative and judicial events related to the migration of non-EU nationals. These statistics complement the statistics based on actual residence under this intervention, and the legal basis was revised only very recently. Statistics on asylum and managed migration are therefore beyond the scope of this evaluation. In addition, European statistics on persons and households based on data at individual level collected from samples are out of the scope of this evaluation, as these are governed by a separate legal basis only recently adopted. Derived statistical indicators in the European System of Accounts and regional statistics based on population counts are also out of scope and will not be evaluated. All these related or dependent European statistics are therefore considered external factors to the intervention evaluated here. The evaluation covers all Member States and EEA/EFTA countries. The time period covered by the evaluation is 2005-2021. This includes the population and housing censuses in the EU of 2011 and 2021. 2.Background to the intervention 2.1.Initial problem/needs statement

Until 2005, to ensure the functioning of the EU in accordance with the Treaties, policymakers needed a variety of statistics for evidence-based policymaking in areas such as: (i) social and economic cohesion; (ii) structural and regional cohesion; (iii) civil rights; (iv) migration and internal affairs; (v) the environment; (vi) energy and (vii) health. In particular, common official statistics for the EU were needed on: ·population and housing censuses (conducted every 10 years) covering population, family, households, and housing at a very detailed territorial level (down to the municipality); ·annual population and demographic events including at national and regional levels; and ·annual international migration at national level, including acquisition of citizenship. In addition to policy needs, EU institutions need high-quality statistical information on population and demography for other purposes. This includes but is not limited to: ·regular, total, usually resident population at national level for voting in the Council; ·regular population projections for EU long-term economic and budgetary projections within the European Semester; ·annual monitoring of the EU’s demographic situation. Figure 1 presents a detailed visual map of the various statistics needs and their policy and institutional drivers. Before the intervention (i.e. before the introduction of the Census, Demography and Migration Regulations), Member States produced all related statistics at EU level on a voluntary basis coordinated by Eurostat. These statistics were produced partly under a soft formalisation through so-called gentlemen’s agreements between Eurostat and the national statistical institutes (NSIs). However, the experience with these arrangements before 2005 showed that this approach could not meet the policy and societal needs at the time. In particular, there were serious and well-known gaps in the completeness, coherence, comparability, and punctuality of statistics disseminated at EU level until 2005. Article 338 TFEU (formerly Article 285 TEC) obliges the legislator to adopt measures for the production of official statistics where necessary for EU policies. Given the shortcomings with this system as it existed until 2005, the Commission took the initiative and proposed legislation to address the gaps in the statistics needed for evidence-based EU policymaking. 2.2.Description of the intervention and its objectives The main reason for legislative action taken after 2005 was the insufficient completeness, coherence, comparability, timeliness and punctuality of the most important topical statistics. Before 2005, these data were produced based on voluntary data collections. Given the experience of weaknesses in the statistics caused by the unregulated nature of the voluntary data collections, Eurostat considered regulation at EU level necessary to address the statistical-quality gaps effectively. The Commission therefore adopted new proposals in 2005, 2007 and 2011 for the Migration, Census and Demography Regulations respectively. These proposals are introduced in Section 1 as the legislative inputs to the Migration, Census and Demography Regulations (see also Section 3). The explanatory memoranda accompanying these proposals outline the policy drivers, at the time the legislative proposal was being drawn up, for the statistical needs identified. Figure 1 sets out a detailed list of these policy drivers. Eurostat prepared the legislative proposals to address the objectives of this intervention in a proportionate but effective way. In particular, the general objectives were to: ·provide sufficient data evidence (sufficient in terms of statistical content and quality) on national and regional population, demography and international migration for EU policymaking; ·address EU institutional needs for statistical information of the highest quality for the functioning of the EU in accordance with the Treaties. The specific objectives were to: ·disseminate complete, comparable, reliable (i.e. accurate, timely and punctual) and regular EU-level statistics on: -persons, families, households, dwellings and housing arrangements from population and housing censuses; -demography, population stock, and population balance;

-international migration and citizenship; -total population at national level for qualified majority voting in the Council; ·provide a sufficient basis of demography and migration statistics to produce population projections for EU long-term economic and budgetary projections; and ·ensure by comprehensive, accurate and comparable metadata including statistical-quality documentation (see Annex 3 for a comprehensive inventory of data collections in each domain). Finally, operational objectives breaking down each specific objective are to provide individual statistical products (e.g. exact cross-tabulations needed for a given statistical unit) serving detailed needs for a given reference time or period. At this level of detail, the three legislative proposals deliberately sought a balance between two sets of objectives: (i) essential operational objectives, to be included as data and metadata transmission obligations in the legal base; and (ii) auxiliary operational objectives, to be implemented by initiating or continuing voluntary data collections outside the legal base. Annex 3 contains a comprehensive tabulation of operational objectives/outputs (datasets). All three legislative proposals upheld a basic principle to ensure proportionality, i.e. to limit regulation to the minimum necessary to deliver on these objectives. This was achieved by proposing to regulate only the essential statistical output (operational objectives) in terms of statistical content and quality. With a legal base adopted at EU level, the targeted quality improvements were to be achieved as follows: ·completeness (of data and metadata across all Member States) through reporting obligations for all Member States on a common set of mandatory data and metadata; ·coherence (across statistical outputs for a single reporting country) and comparability (across reporting countries for a single statistical output) through legal provisions on common statistical definitions, topics and breakdowns, accompanied by harmonised methodological guidance in each domain; ·timeliness (reducing the time between the end of the reference period and dissemination of a statistical output) and punctuality (reducing the time between the scheduled and actual dissemination of a statistical output) through legally fixed common transmission deadlines for all mandatory data and metadata collections. However, such firm output orientation entails particular need for comprehensive quality reporting across the entire statistical production process (the last specific objective). This was reflected in dedicated provisions on mandatory metadata and/or quality reporting across all proposals. 2.3.Baseline and points of comparison, including evolution of policy needs Most statistics included in this intervention had already been collected on a voluntary basis before 2005. The situation before 2005 – i.e. the content and quality of all relevant statistics disseminated at EU level until the first reference periods of the legal bases – is thus the baseline to which the post-intervention situation should be compared. In all three statistical domains (the population and housing census; demography statistics; and international migration statistics), the pre-intervention situation is given by all public data and metadata available on Eurobase for the respective domain-specific pre-regulation periods. The post-intervention situation is given by all public data and metadata on Eurobase for the respective domain-specific regulated periods up to and including 2019 (see the concise overview in Table 1 and detailed data collections to be traced back across the points of comparison in Annex 3). The pre- and post-intervention situations are also helpful points of comparison for determining the problem/needs definition. More precisely, two reference times are fixed: (i) the pre-2005 situation establishing the original policy drivers that led to the statistical needs identified for the intervention; and (ii) the post-2021 situation anticipating evolved policy drivers for the medium-term future after the 2021 census round. This comparison will be a key element of this evaluation, enabling an updated assessment of the relevance of the intervention in various policy contexts that experienced a great deal of change over the past decade (see Figure 1 outlining the evolution of broad policy drivers between 2005 and 2021; details are presented under the relevance assessment in Section 5). Therefore, a related key element of this evaluation is a detailed assessment of the voluntary data collections that were initiated in parallel to the implementation of this intervention, often to address ad hoc urgent policy needs for statistics. These include, for instance, data collections on new migration flows after the United Kingdom left the EU and new infra-annual statistics responding to the COVID-19 pandemic. All of these ad hoc voluntary data collections are also included in Annex 3. Finally, there is a general lack of sufficiently detailed background information on this intervention. This is because requirements for impact assessments at the time (before 2005) were far less detailed than they are today. For this reason, significant information gaps were encountered when attempting to reconstruct a comprehensive and reliable picture of the baseline (the pre-2005 situation and initial assumptions determining the intervention). Therefore, the default baseline is sometimes complemented by specific accessible points of comparison – for example a hypothetical scenario of perfect statistical quality – when evaluating certain quality indicators.

3.Implementation/state of play A comprehensive overview of the intervention logic and its implementation steps down to current results and impacts is provided in Figure 2. 3.1.Inputs and external factors As noted in Section 2, the intervention logic is closely linked to the legislative initiatives put forward by the Commission. The Commission considered the legal acts (i.e. the Census, Demography and Migration Regulations) to be the key instruments for producing outputs (high-quality official statistics and quality documentation) that deliver on the objectives of the intervention. Therefore, all legal acts adopted under the intervention are considered as inputs for this evaluation. Figure 2 contains a comprehensive list of these inputs. In this context, Regulation (EC) No 223/2009 on European statistics (the ‘European statistics Regulation’) extends the mentioned legislative inputs by providing the general legal base for a common quality framework for all official statistics produced at EU level, including all domains evaluated here. The resource inputs to the intervention were the human and financial resources at both EU level (mainly Eurostat staff and calls for grant proposals organised by Eurostat) and national level (mainly staff of NSIs and national funding for the data collections). The adoption of the legal bases created legal obligations that ensured the stable availability of these resources during the implementation of the Census, Demography and Migration Regulations. A quantification of these resources will be a central element of the evaluation method described in Section 4. Various external factors provide important context – either to the implementation process or to the outputs of the intervention. These external factors are addressed in the paragraphs below. Related European statistics are those official statistics disseminated by Eurostat that are similar enough to the domains addressed by this intervention to raise problems of coherence and interdependence. However, they are conceptually or methodologically distinct enough that they were considered out of scope of the intervention. Among the most relevant domains are asylum and managed migration statistics, which are mostly regulated under Articles 4-7 of the Migration Regulation and deal with administrative and judicial procedures related to migration (e.g. asylum applications and decisions, Dublin procedures, enforcement of immigration legislation, and residence permits). There are close conceptual links to the statistics considered under this intervention, but the relevant statistical units are profoundly different: resident population of a given geographic area and its changes (this intervention) versus migration-related administrative or judicial acts. Furthermore, samples of persons and households generally depend on the full-enumeration population statistics under this intervention (sampling frames). They thus share many statistical concepts and definitions of this intervention, but on the other hand they are fundamentally different from a methodological perspective. The recent adoption of Regulation (EU) 2019/1700 on European statistics relating to persons and households, based on data at individual level collected from samples (footnote 9) has created a new context for the population statistics evaluated in this staff working document. Finally, national accounts use population as well as household and dwelling figures from censuses to produce derived indicators. International statistical cooperation creates an environment of common concepts and definitions in which many ESS members participate. Eurostat actively contributes to – and promotes – this environment at various levels to foster the international harmonisation of official statistics. Key aspects of this environment are the statistical coordination and governance activities coordinated globally by the UN Statistical Division and for the European region of the United Nations by the UN Economic Commission for Europe (UNECE). Networks of experts organised by these United Nations bodies produce and regularly update international guidelines (such as the Recommendations for the Censuses of Population and Housing endorsed by the Conference of European Statisticians). Demographic research activities at EU and international levels have been an element of attention during the period of implementation of this intervention. One demographic research activity has been in particular focus recently: the concept of actual presence. This concept is related to the usual-residence concept, and has been the focus of recent research using new data sources to measure migration and cross-border mobility, such as mobile devices, social media networks, satellite images and internet platforms. Another area of research where Eurostat is an independent actor is the harmonised population projections for the whole EU based on European demographic data. Eurostat corresponds regularly with the international community in this area and with ESS partners. Eurostat projections are a key input to EU long-term economic and budgetary projections mandated by the Council (footnote 11). 3.2.Description of the current situation Based on the above inputs and external factors, Member States engaged in several activities to implement the three Regulations comprising the intervention. The first activity by Member States was the development and operationalisation of mandatory data and metadata collections under the newly adopted legal bases. Eurostat also took action, further processing the data collected from Member States to help the dissemination of European statistics. Eurostat’s action in this area focused in particular on data validation and quality assurance, as well as on methodological guidance where necessary to implement common statistical concepts coherently in national data collections. Finally, Eurostat also collaborates at EU level and internationally including with statistical producers, policy users, EU institutions and international organisations (see Section 4). The implementation of the intervention can be considered as completed to the extent that all mandatory data collections have now entered a mature phase characterised by routine and robust production processes that have now been established for some years. When necessary, these processes continue to undergo mostly occasional, technical or small methodological updates. This is illustrated by the generally very good compliance of national statistics providers with the legislation in force, as documented in regular implementation reports to the European Parliament and the Council for the annual data collections. Eurostat monitors legal compliance regularly, and this led to some bilateral exchanges with some Member States at technical level, but outputs generally achieve a high level of compliance. The statistical outputs being generated by the intervention have resulted in tangible benefits for statistics users. Firstly, the statistics help to inform evidence-based EU policymaking in many areas such as: (i) social and economic cohesion; (ii) structural and regional cohesion; (iii) civil rights; (iv) migration and internal affairs; (v) environment; (vi) energy; and (vii) health. Moreover, total population at national level (based on a strict application of the twelve-month definition of usual residence, reported under Article 4 of the Demography Regulation) is used to verify the population quota in qualified majority voting of the Council. Furthermore, annual demographic statistics and projections inform EU institutions to ensure the functioning of the EU. For instance, these statistics and projections contributed to the 2021 Ageing Report and are a basis for the EU’s long-term economic and budgetary projections under the European Semester. They were also used for the 2020 Report on the Impact of Demographic Change addressing Article 159 TFEU (formerly Article 143 TEC). Finally, statistical outputs from this intervention are among the most widely consulted European statistics. Since 2016, international migration statistics have been the most visited of the thematic sections on the website, with strong and continuing user interest. Similarly, the periodically updated ‘Statistics Explained’ article on migration and migrant population statistics has established itself since 2016 as one of the most frequently consulted ‘Statistics Explained’ pages. However, there are two substantial developments that could not have been foreseen at the time the legal bases for the intervention were being drawn up. The first of these developments is the rapid change in relevant policy drivers over the past decade. There has been particularly rapid change brought by: (i) demographic developments including migration and ageing; (ii) the Green Deal; (iii) fundamental rights monitoring; and (iv) the urbanisation and integration of rural areas. This created a situation where statistics producers, who were still implementing relatively recent obligations under the intervention, were confronted with new and quickly changing policy needs for statistics. Often these needs were out of scope of the recently enacted legal bases. The second development is the equally rapid change in the relevant data sources, a change that was also not anticipated by the legislation. There were three aspects to this rapid change in data sources: (i) the rapid improvement in the quality and accessibility for statistical purposes of administrative sources available in an increasing number of Member States; (ii) the establishment of large-scale IT systems and interoperability platforms at EU level; and (iii) the emergence of ‘big data’, including privately held data sources. As a result of these two main developments, new or changed policy needs were emerging in parallel with developments in data sources and methodologies that could not be properly exploited within the legal framework. In conclusion, the current situation is characterised by a firm legal framework that emerged from the intervention, and very good overall compliance by all Member States with this framework. This evaluation should therefore focus on the legal framework’s ability not only to deliver on the initial objectives of the intervention, but also to remain relevant in a highly dynamic policy, technical and methodological context. 4.Method

4.1.Short description of methodology This evaluation assessed the performance of the current legal framework on European population statistics according to six criteria: relevance (2005 situation and evolution until today), effectiveness, efficiency, coherence (internal and external), EU added value and statistical quality. To achieve this, and to answer the evaluation questions in Section 5, Eurostat assessed two broad perspectives: (i) how the statistics are produced from source to publication; and (ii) how these publications are received by the statistics users. The full evaluation framework (criteria, questions, indicators and evidence sources) is provided in Annex 4. The initiative on redeveloping European statistics on population (ESOP) was published in March 2021. It relies on a back-to back evaluation and impact assessment. The combined evaluation roadmap and inception impact assessment was publicly consulted in April 2021. Eurostat then carried out the evaluation, impact assessment and stakeholder consultation between May 2021 and February 2022 with the support of an external contractor and guidance from an interservice steering group (ISG) composed of representatives of 16 Commission Directorates-General (DG) (see Annex 1, point 2). Eurostat identified the following evidence and corresponding sources to be used for evaluation and impact assessment: ·desk research – analysis of legal, contextual, methodological and technical background documents relevant to the intervention; ·opinions of statistics producers – various consultation activities engaging NSIs, national holders of administrative source data used for population statistics, and EU citizens acting as census respondents; ·opinions of statistics users within the Commission – consultation activities and drawing upon Eurostat’s working relations and good communication with many Commission services and in particular the Commission network of statistical correspondents; ·opinions of other users – various consultation activities, in particular the open public consultation (OPC). A detailed account of all evidence and sources used can be found in Annex 1.5. Eurostat then developed a consultation strategy that further refined the typology of key stakeholders as well as suitable consultation activities. The following main stakeholders were identified. ·Data providers comprising administrative data providers (public administrations and other organisations that provide or may provide source data to statistical authorities for producing European statistics) and individual respondents (individuals who are included in the data collection such as sample surveys and census enumerations and who provide answers to the statistical authorities collecting this data). ·Statistics producers are the NSIs and other national authorities collecting, processing and sending relevant statistical data to Eurostat. They ensure the quality of the data. ·Statistics users, divided into four categories set out below. oInstitutional users are the policymakers or entities directly supporting policymaking at various administrative levels. They include EU bodies, international organisations, national ministries, government research institutes, and regional/local authorities. oOther professional users contribute occasionally and indirectly to the policymaking process at EU and other levels. These other users include academics, research institutes, professional organisations, advisory councils, NGOs, and individual private companies. oMedia organisations. oThe general public. The stakeholder groups were then mapped onto suitable and proportionate consultation activities in a detailed consultation plan (Table 2). Eurostat carried out the expert consultations in its relevant expert groups with ESS members (the Directors of Social Statistics group, topical working groups, and a task force) and by written consultations, for instance with the Commission network of statistical correspondents. Eurostat also prepared and implemented the public consultation survey and two targeted consultation surveys partly with support from the contractor. Finally, 5 workshops and 47 in-depth interviews were organised, hosted and documented by the contractor with Eurostat inputs on the selection of appropriate participants, agenda setting and occasional interventions. Due to the back-to-back evaluation and impact assessment, all consultation surveys contained both: (i) sections or elements focusing on an evaluation of the current framework (a ‘backward-looking’ assessment – analysed in this staff working document); and (ii) ideas for future improvement (forward-looking – analysed in the impact assessment). Analysis of the two targeted surveys was straightforward, as these mapped well onto specific user subgroups listed above, namely EU-level institutional users (targeted at the network of statistical correspondents of all Commission services) and statistics producers (targeted at the NSIs). To analyse the results of the OPC, all replies were first categorised into the key stakeholder groups listed above and then assessed in turn. The workshops were used to obtain stakeholder feedback at the start and at the end of the evaluation (addressing the problem definition – September 2021, and the problem validation – January 2022). 4.2.Limitations and robustness of findings

A broad structural limitation of the desk research was the fact that the basic legal acts under this intervention were adopted between 2007 and 2013 – at a time when the Better Regulation guidelines and ensuing need for effective monitoring and evaluation mechanisms were not in place yet. Moreover, the costs of producing statistics at Member State level were not documented in sufficient detail at that time. In particular, virtually all national production systems for population statistics were historically set up, maintained and updated to simultaneously cater to both national and EU-level statistical needs. This makes it extremely difficult for Member States to separate: (i) the baseline costs of catering to national needs; from (ii) the incremental costs incurred only to comply with the legal framework under this intervention (and costs of catering to EU-level statistical needs more generally). Finally, the period when the legal framework was implemented in the Member States – roughly from 2007 to 2014 – coincided with another fundamental trend, namely the first stages of the transition of many national statistical production systems from traditional data sources to an increased – or even exclusive – use of administrative data sources. This broad trend is assumed to have substantially affected costs across this period. This cost effect is likely to have been much greater than any changes in costs due to the intervention – at least in some Member States. These limitations have made it difficult to quantify implementation costs cleanly against the hypothetical baseline (i.e. the 2005 situation augmented by the effects of Regulation (EC) No 223/2009), and thus to assess efficiency quantitatively. Therefore, related evaluation questions and indicators were designed to acknowledge this difficulty by adding additional qualitative opinion elements covered by the stakeholder consultation. Nevertheless, monetised baseline and incremental costs on Member States and on Eurostat itself by each statistical domain, are estimated from the consultation results at hand, combined with both regular Eurostat consultations on statistical production costs and UNECE publications on the costs of previous census rounds. The estimation models and assumptions are described in detail in Annex 2, and findings are presented under the discussion of efficiency in Section 5.4 below. It was not possible to quantify or estimate the monetary value of: (i) costs for data users or for individual persons; and (ii) benefits in general. This was in part due to a lack of available data, for example on the costs to citizens of participating in census rounds. More generally, this was due to certain costs and benefits being inappropriate for quantification because their effects are more ambiguous and variable across Member States and stakeholder groups. For example, the benefits to non-institutional data users from increased access to high quality European statistics on population would be challenging to quantify, since this would depend on several additional factors, such as how these data would be used or the cost of accessing data through alternative sources. 5.Analysis and answers to the evaluation questions This evaluation addresses all five standard criteria (relevance, effectiveness, efficiency, coherence, and EU added value) as well as statistical quality as an auxiliary criterion. Statistical quality affects the standard criteria in various ways, so that indicators contributing to the standard criteria often draw from statistical quality indicators as part of the evidence. In general, each evaluation question was assessed with at least one indicator, where indicators are either qualitative or quantitative depending on the source data available and the nature of the indicator. An overview of the complete evaluation framework in terms of criteria – questions – indicators is provided in Annex 4. 1. 2. 3. 4. 5. 5.1.Relevance – initial situation (RI) The need for this intervention was based on statistical requirements and constraints that existed in the baseline scenario of around 2005. However, the reference scenario has changed significantly between the baseline and today. Therefore, an evaluation of the relevance of the intervention should take at least two perspectives: performance against the baseline scenario (‘initial relevance’) and performance against current needs and constraints (‘evolved relevance’). This section evaluates only initial relevance, whereas evolved relevance under the reference scenario of today is evaluated in section 5.2. RI1 – To what extent do statistical objectives and outputs correspond to the needs for evidence-based EU policymaking? → All EU institutional- and policy-related topical needs known at the baseline were addressed. When the intervention began around 2005, several EU-level data needs in the population-statistics domain were well known and expressed by various EU bodies (footnote 1). These needs are documented in the explanatory memoranda of the Commission proposals for – as well as in the recitals of – the current legislation. As Figure 1 shows, the EU-level needs can be broadly split into policy needs (supporting EU policies) and institutional needs (emerging directly from provisions in the Treaties explicitly relevant to population statistics). Indicator RI1.1 measures if/when initial EU use cases (policy and institutional) were addressed successively by datasets that became regulated through this intervention. Figure 3 shows the time evolution of the indicator, clearly highlighting the step-by-step improvements around the years when particular base acts started to deliver data (2009 for the Migration Regulation, 2014 for the Census and Demography Regulations). The key quality gaps at the time (2005) that were addressed by this intervention were comparability, timeliness and completeness of the data across all Member States (see Section 5.8). RI2 – To what extent do statistical objectives and outputs serve institutional needs for the functioning of the EU? → Addressing EU institutional needs for the definition and implementation of the population base leaves room for improvement. Gaps exist in statistics inputs for population projections. There are three main institutional needs introduced in Section 2 for European population statistics: (i) monitoring of the demographic situation; (ii) national population weights for qualified majorities in the Council; and (iii) inputs to population projections needed for long-term economic planning. These needs imply certain aspects of the statistics: for instance, policy and democratic-representation considerations require a population-base definition (who is counted among the population and who is not) that reflects the actual population present. Indicator RI2.1 shows that the compromises accepted under this intervention have generally led to a regulated definition of the population base that is overall barely sufficient in this respect. The Demography Regulation requires national population figures based on a strict base definition only for Council voting weights. These national population figures must be of the highest possible quality but the factual accuracy or comparability achieved is currently not quantified, and there are indications that the strict definition is not implemented coherently in all cases (RI2.2, see EE3.1). Finally, there are gaps in statistics inputs for producing population projections that were in principle known at the time of developing the intervention, but not fully reflected in the resulting legal framework (RI2.3).

5.2.Relevance – evolution until today (RE) RE1 – To what extent do population statistics address current policy needs for detailed, frequent and harmonised data on population aspects, including at highest geographic granularity? → The evolution of EU policy needs since the baseline has led to significant new data gaps. The previous section showed that the intervention has indeed led to many highly relevant and needed datasets from a baseline perspective. However, indicators RE1.1–RE1.5 measure the gaps due to emerging policy needs that currently exist but that were not anticipated at the baseline. Indicator Gap type Baseline Target Current Key gap(s) RE1.1 Detail Good Good Barely sufficient -Housing data for Green Deal -Equality data for fundamental rights policies RE1.2 Frequency Good Good Decent -Housing data for Green Deal -Quarterly (seasonal) population for urban/rural integration RE1.3 Timeliness Good Good Decent Legal deadline for EU census results of 27 months after the census year is too late RE1.4 Harmonisation Decent Good Barely sufficient No harmonisation of the population-base definition at EU level RE1.5 Geographic granularity Good Good Barely sufficient NUTS-level data, incl. on migration, functional typologies and georeferenced data for regional and urban/rural cohesion policies, and cross-border analysis In summary, the evolution of the policy environment over time has led to significant gaps that did not exist at the time of the baseline, most notably in the detail of characteristics, harmonisation, and the geographic granularity of the statistics. RE2 – To what extent are established statistical objectives fit to respond to evolving policy needs? → RE1 identified a significant increase in gaps over time. Statistics users across all groups do not find that the statistics adapt quickly to new needs. The previous question identified several key gaps that emerged after the intervention was implemented. According to indicator RE2.1, the current legal framework is now barely sufficient to address these gaps, essentially because it provides almost no flexibility mechanisms (e.g. only very few details can be changed by delegated powers). This is substantiated by the OPC results, where most respondents across all stakeholder groups except statistics producers agreed only ‘somewhat’ that the legislation is fit for purpose. The inability of the legal framework to adapt is also illustrated in the clear opinion of statistics users that the statistics cannot adapt quickly to new needs. Only a minority of OPC respondents across all stakeholder groups (except statistics producers) agreed that the statistics adapt quickly to new needs. The share of disagreement was most pronounced among the professional and institutional users. Moreover, opinions on the ad hoc data collections initiated outside the legal framework in response to COVID-19 seem to indicate that such voluntary ad hoc measures may not be sufficient to meet the needs experienced in a rapidly changing crisis. Finally, after the 2011 census, Eurostat and the NSIs concluded there was a substantial need for a complete collection of key 2021 census data on a common European 1 km2 grid. However, this could not be regulated under the Census Regulation due to its lack of flexibility. Therefore, the Commission had to proceed with an ad hoc act under a different legal base to ensure EU completeness and comparability. RE3 – Who are the main current users of European statistics on population and to what extent do the currently available European statistics on population meet their needs? → A variety of user groups other than EU institutional users is identified. These groups are generally somewhat more satisfied with the current statistics, but raise the same main topics for improvement as RE1. RE1 focused on the needs of EU-level institutional users representing the most influential user group of European statistics. However, in addition to institutional users at other governance levels, the stakeholder consultation also addressed other relevant user groups in line with the expectations of the consultation strategy. These other relevant user groups most notably include NGOs active in the relevant policy areas and academic/research institutions. The views of these user groups are captured by the OPC, workshops and interviews. The consultations of these more diverse groups found that they are generally somewhat more satisfied that their data needs are being met than EU-level institutional users. However, these more diverse groups have also identified areas for improvement that are in line with the findings of RE1, namely: (i) the timeliness of statistics; (ii) the availability of data on subgroups at risk of inequality and non-discrimination (such as LGBTI groups and ethnic minorities); (iii) the availability of data on EU internal and external migration; and (iv) access issues (such as the utility of metadata and other user-friendliness concerns). This is also reflected in Figure 4, which shows OPC priorities for dataset topics to be improved by 2030. Thus, changes in the needs of statistics users often reflect major societal changes such as migration, gender issues and social norms. 5.3.Effectiveness (EE) EE1 – To what extent is the output of high quality? → Several quality aspects have improved significantly, especially for statistics that were included in the three new Regulations. Key gaps remain in: (i) availability of statistics when data transmission is voluntary; and (ii) comparability of statistics due to insufficient harmonisation. At the time of the baseline around 2005, all relevant statistics were produced on a voluntary basis. This led to significant gaps in the statistical-quality dimensions of comparability, timeliness and completeness at EU level – i.e. data were not available for all Member States, or were only available after considerable delay. It also meant that some statistical categories were not fully comparable. Therefore, improvement on these quality aspects is a criterion for measuring the effectiveness of the intervention. Findings from indicators on respective quality dimensions are summarised in the table below. Quality dim. Ind. Definition Baseline Target Current Coherence SQ1.3 Percentage of ESS members with coherent annual population totals across datasets 72% (2005) 100% 100% (from 2014 on) Coherence SQ1.5 Percentage of ESS members with coherence between annual population stocks and demographic changes 44% (2004-2012 av.) 100% 43% (2013-2019 av.) Comparability SQ2.2 Percentage of consistent bilateral migration flows between ESS members (<20% difference) 25% (2005) 100% 25% (2019) Comparability SQ2.3 Relative asymmetry of total intra-EU migration (% of total immigration) 13.8% (2008) 0% 3.7% (2019) Timeliness SQ4.1 Largest delay of EU complete annual population data compared to reference date 552 days (2007-2012) ≤ 552 days 397 days (2013-2019) Punctuality SQ5.1 Largest delay compared to agreed deadline among ESS members 294 days (2007-2012) 0 days 31 days (2013-2019) Completeness SQ6.1 Percentage of mandatory statistics published with EU-level completeness 42.6% (2000-2006 av.) 100% 98.9% (2013-2019 av.) Completeness SQ6.2 Percentage of voluntary statistics published with EU-level completeness 41.2% (2000-2006 av.) 100% 58.4% (2013-2019 av.)

This shows that coherence (SQ1.3), comparability (SQ2.3), timeliness (SQ4.1), and completeness (SQ6.1) have all improved significantly for data that became mandatory with the intervention. However, the quality aspects of voluntary data have not improved at the same rate (SQ2.2, SQ6.2), which was to be expected given the baseline experience. This is very much in line with the general opinion of respondents to the public consultation, who largely agreed (more than 75% agreed in each stakeholder group) that the current statistics are of high quality overall. There was also widespread agreement that the voluntary parts should be subject to a regulation in the future (more than 75% in each stakeholder group agreed except for the statistics producers). Finally, the lack of improvement on indicator SQ1.5 – despite underlying mandatory data – is a direct consequence of the harmonisation gap identified in question RI2 that persists even after the intervention. EE2 – To what extent do statistics published under the intervention serve EU policymaking? → The availability of EU aggregates and complete coverage of the EU population and its demographic characteristics have generally improved, leading to improved evidence for EU policymaking. A key element of data utility for policy-making at EU level – and thus effectiveness of this intervention – is the availability of EU aggregates. To make EU aggregates available, the data need to be complete across all Member States and a dedicated aggregate category must be created in the geographic breakdowns – e.g. ‘EU28 (2013-2020)’. The left-hand side of Figure 5 shows the average completeness of EU aggregates in annual datasets, illustrating two trends. First, the Migration Regulation introduced various mandatory or partly mandatory datasets from 2009, but without including clear approaches to aggregate migration data coming from NSIs to EU level. Second, the Demography Regulation introduced mandatory datasets and required EU aggregates to be calculated (a significant improvement can be seen from 2013). Another key element for policymaking at all levels is the comprehensive coverage of the population under the required characteristics. The presence of ‘unknown’ categories reduces this comprehensive coverage for the affected variables. The right-hand side of Figure 5 shows the average percentage of the population with an ‘unknown’ characteristic across all affected variables and Member States in the annual data. Again, there is a clear performance difference between the mandatory and voluntary datasets. In particular, legislation to implement the Demography Regulation capped the population percentage reported as ‘unknown’ to at most 5% of the total population in the mandatory data. The sharp increase in the voluntary data around 2009 comes from the introduction of a variable on education that was initially reported as ‘unknown’ to a significant extent. EE3 – To what extent do statistics published under the intervention serve institutional needs for the functioning of the EU? → There are indications that population figures for Council voting weights can be made more coherent for all Member States. Although core topics for demographic monitoring are covered well, data gaps have opened up on emerging topics of demographic and societal change. As discussed under question RI2 (Section 5.1), the quality – and in particular the accuracy – of total population figures corresponding to a strict common definition of the population base (usual residence based on 12 months stay) is central to the effectiveness of these data to establish reliable Council voting weights. Indicator RI2.2 already established that there is insufficient information available on accuracy. Therefore EE3.1 analyses the coherence in implementation of the strict definition across Member States and thus the emerging comparability of these figures at EU level. The information available on national situations indicates that 22 Member States applied the definition coherently in 2022, up from 17 Member States when this data collection started in 2014. Some progress has therefore been made and this has led to decent EU-level comparability of these figures at present, even though a major gap remains due to a lack of information on accuracy. The appropriateness of the statistics for monitoring the demographic situation is another factor in the effectiveness of statistics for institutional purposes. A ‘soft’ indication of its performance in this respect can be seen in the opinions gathered in the OPC on whether various elements of demographic and societal change are sufficiently covered by the current statistics (EE3.2). Figure 6 shows several topics where only a minority of the respondents from the institutional user group found the currently published statistics to be sufficient. These topics are equality (only 31% of respondents found the statistics sufficient), housing (33%), urban/rural mobility (33%), intra-EU migration (38%) and rural population (47%). These responses are in line with the targeted consultation with Commission statistical correspondents, who said that the key policy topics suffering from insufficient data were: the Green Deal (housing data), urban/rural integration (functional typologies and seasonal data) and fundamental rights policies (equality data). Among all other professional users – including researchers – the picture is even more distinct, with even smaller shares than institutional users saying there was sufficient data on equality (14%), housing (21%), urban/rural mobility (28%), intra-EU migration (37%) and rural population (42%). The other professional users were also mostly dissatisfied with the data on acquisitions or losses of citizenship (only 30% were satisfied) and urban population (41%). EE4 – What are the existing cooperation arrangements between NSIs and national authorities in charge of administrative data sources that are used for population statistics? How effective are those arrangements? → Cooperation arrangements between NSIs and national authorities are largely effective, although stakeholders identified clear room for further improvement. The targeted NSI survey asked about the existence of legislative or other formal arrangements for cooperation between the NSI and the national authorities in charge of relevant administrative sources. In their responses, NSIs pointed to written agreements with the data owners (79% of NSIs said they had such agreements in place for all or most relevant sources), followed by effective coordination mechanisms in place (59%). Over a third of participating NSIs indicated that they are informed in advance of changes to administrative data sources for all or for the most relevant sources (34%), and 28% said they were consulted on designs or changes to administrative data sources. These results tend to match the results from the case-study interviews, indicating that NSIs have a series of coordination mechanisms that can be activated for different types of sources. Nonetheless, the workshops showed a continuous discussion on the difficulties that NSIs face in accessing data. In the NSI survey, NSIs were also asked how often they faced difficulties in cooperating with data owners. Most NSIs (66%) said they rarely face difficulties. Less than a quarter said they faced issues always or sometimes (24%), while 10% said they never face issues. When asked about main limitations/restrictions in the cooperation agreements, the main issues raised by the NSIs were excessive formality or bureaucracy. The NSIs also said that delays in sending data were generally a rare occurrence, and outright refusal to send data was either rare or did not happen. Nevertheless, most NSIs (68%) agreed that there was room to improve cooperation between NSIs and national authorities, while 21% disagreed. Findings from the case studies indicate that, overall, the current agreements seem to be sufficient to gather the required data, but not sufficient for gathering data on areas of emerging interest. Although the situation may vary between countries, these results seem to follow a general trend, namely that serious problems with data access only seem to affect specific sectors. 5.4.Efficiency (EI) EI1 – To what extent is the output compliant with legal requirements? → Only occasional issues were followed up on and no infringement cases were launched. The current legal base consisting of three separate acts leads to administrative inefficiencies (REFIT relevance). Since the adoption of the three base acts, Eurostat has monitored the legal compliance of Member States with mandatory statistical requirements as a part of its implementation efforts, but being somewhat less strict during the transitional periods after new obligations entered into force. Overall, Eurostat only identified occasional issues in compliance over the years, and these issues were followed up at expert level to improve compliance gradually. Eurostat monitored and addressed compliance, and did not have to launch any infringement procedures in the process (EI1.1). Another issue relevant to efficiency is the fact that the current legal framework consists of three separate legal acts that Member States need to be comply with and that Eurostat needs to enforce and monitor. Apart from trailing internal incoherence (Section 5.5), this is assumed to cause at least some redundant administrative procedures, creating opportunities for REFIT simplifications.

EI2 – To what extent are voluntary data collections required to cover statistical needs? → The intervention was efficient in regulating a large part of the statistics addressing key needs. Nevertheless, there is room for further efficiency gains due to the existence of many currently voluntary datasets that are highly complete (REFIT relevance). Voluntary data collections are generally less efficient at EU level, because not all Member States invest the same resources in providing these data. This means that most of the EU-level value added is lost because these data are then not complete, timely and/or comparable enough. Often, a relatively small incremental investment in completing the data, i.e. by regulating them and supporting implementation financially, can lead to significant increase in value added. Therefore the share of data needs covered by voluntary collections (EI2.1) is a proxy indicator for the efficiency of the intervention. Figure 7 shows the evolution over time of the numbers of mandatory, partly mandatory, and voluntary datasets published. Three changes are obvious from this figure. The first change is the introduction of mostly partly-mandatory datasets by the Migration Regulation from 2008 on. The second change is the conversion of voluntary datasets to mandatory datasets, and the addition of new voluntary datasets in the wake of the Demography Regulation from 2014 on. The third change is the sharp increase in voluntary datasets from 2020 responding to urgent needs related to COVID-19. Thus in 2020, almost half of the annual and infra-annual datasets published in the ESOP domains are voluntary, and only a third of all datasets are fully regulated by EU legislation. This indicates room for efficiency gains, also in a REFIT context, especially by regulating voluntary datasets that are already now highly complete. EI3 – How often are mandatory vs voluntary datasets accessed by users? → In line with the general trend, the number of queries of population statistics has roughly doubled over the past decade for most domains. Mandatory datasets are significantly more popular than voluntary ones. OPC respondents from almost all stakeholder groups use the Eurostat website or database to consult statistics on the European population. The only group with a significant minority (41%) that did not know about these resources were the individual respondents to statistical survey questionnaires (i.e. non-regular users). On access to annual data through Eurobase (EI3.1), Figure 8 (left-hand chart) shows that the numbers accessing the most popular demographic dataset (demo_gind – demographic balance) have increased significantly over the past decade (from less than 50 000 per year until 2014 to more than 100 000 per year since 2018). However, this increase in interest is not unique to population statistics and therefore cannot be attributed to an isolated effect of the intervention. For example, popular Eurostat datasets across other domains (e.g. GDP, the consumer price index, and unemployment) have also seen an increase in popularity, roughly doubling access numbers over the past decade. The right-hand side of Figure 8 confirms that the partly or fully regulated datasets under this intervention are also significantly more popular on average than the datasets that remained voluntary. On access to 2011 EU census output through the Census Hub (EI3.1), Eurostat did not produce regular analytics. An ad hoc extraction was only possible from 2017 onwards, so there are no data for the period most likely to have seen the most user interest, i.e. during the period after initial publication in 2014. Nevertheless, the website analytics for 2017-2021 show that 2011 EU census outputs are still just as popular as annual data, with more than 160 000 queries per year for the most popular dataset (hypercube 55 – population by NUTS 3 region, sex and single year of age). And on average, there are almost 8 000 queries per year and per hypercube on population (more than 6 300 queries on housing, 2 100 on families and fewer than 1 000 on households). This also shows that census outputs on housing are almost as popular as outputs on population. EI4 – How efficient are existing cooperation arrangements between NSIs and national authorities in charge of administrative data sources used for population statistics? → Even though cooperation arrangements are widespread, NSIs attribute efficiency losses mainly to the insufficient willingness of source owners to communicate and involve NSIs in the design of – or changes to – these sources. Good and close coordination seems a key factor. EE4 revealed a problem: although there are already formal, effective cooperation arrangements between NSIs and national authorities, the role of most NSIs is more limited in decision-making or change processes related to the relevant administrative sources. For instance, 66% of respondents to the targeted NSI survey noted that they were informed in advance of changes to administrative sources only for some or none of the relevant sources. And 72% of respondents to the targeted NSI survey said they were consulted on the design of – or changes to – only some or none of the relevant sources. Therefore, despite formalised – and to some extent efficient – cooperation mechanisms, some NSIs experience insufficient communication or involvement in key decisions made by national authorities. This may lead to inefficiencies, as substantiated by feedback from interviews, where views on cooperation arrangements varied greatly across Member States. Some of the key challenges and drivers of inefficiencies in these cooperation arrangements are set out in the bullet points below. ·The NSI is rarely involved in making decisions on data structure and quality-related issues, including when designing or modifying administrative databases. ·There is often a lack of established cooperation mechanisms between NSIs and data owners. ·Data owners are often not willing to consider recommendations or requests from NSIs, and NSIs often lack the authority to request changes. ·Excessive bureaucracy and delays often occur in the delivery of data, particularly sensitive or protected data. ·In some cases, NSIs reported that data owners misinterpreted national law and refused to provide access to data and/or requested payments from NSIs for the data. ·NSIs that reported strong collaboration arrangements attributed this to the existence of established coordination mechanisms, including written agreements covering each instance of data provision with data owners. When asked to identify areas of cooperation that required improvement to achieve cost savings in the production of population statistics, NSIs identified the following key areas: (i) the quality of data; (ii) staff availability; and (iii) communication between NSIs and national authorities. A substantial proportion of NSIs (48%) were unable to provide a response when asked for suggested improvements to cooperative arrangements to ensure European statistics on population are able to meet user needs. Those NSIs that did respond gave suggestions relating mainly to data access, data sharing, and the frequency of data sharing.

EI5 – What costs do data producers currently face when developing European statistics on population? → The costs of the intervention are mainly borne by Member States’ administrations and Eurostat. Due to implementation of the intervention, the recurring baseline costs for statistical production have increased by roughly 20% for Member States and by roughly a factor of 5 for Eurostat, with the census accounting for the biggest share, although this depends greatly on the census method adopted by the Member State (register-based, traditional or combined). The analysis by the contractor suggests that, overall, the main costs associated with the development of European statistics on population have been borne by: (i) NSIs/Member States; (ii) Eurostat; and (iii) to some extent the general population. However, it was not possible to quantify these costs and therefore they are not discussed in detail in this analysis. The greatest costs were incurred by Member States and their NSIs. The types of costs included in the cost analysis conducted as part of this study are detailed in Annex 2. These are set out in the two bullet points below. ·Costs to Eurostat: regulatory costs (for preparing/communicating the legislation and providing financial support), enforcement costs (or costs relating to monitoring compliance with the legislation and publishing data) and IT costs. ·Costs to NSIs or Member States: compliance costs (relating to monitoring and ensuring compliance with the legislation), administrative costs (to design and implement procedures to collect/distribute data, as well as to train staff), enforcement costs (covering costs required to monitor and report on data provided e.g. in terms of data quality etc.) and IT costs. As set out in Table 3, the analysis of costs developed as part of this evaluation generated an indicative estimate of the incremental costs to NSIs/Member States and to Eurostat associated with the current regulatory framework. This indicative estimate is relative to the baseline scenario before the implementation of the Census, Migration and Demography Regulations. Table 3 sets out the sum of costs associated with all three Regulations. Overall, additional costs to all Member States and their NSIs are estimated to be around EUR 58 million on average every year (an average of 2.2 million per Member State), with an additional EUR 4 million in one-off set-up costs. This suggests that costs to Member States increased relatively little (by around 20%), adding to the existing baseline cost of EUR 270 million. For Eurostat, incremental costs are estimated at EUR 1.1 million on average every year, with a EUR 1.2 million additional one-off cost. These estimates suggest that, relative to the baseline, costs to Eurostat more than quadrupled, in line with its increased legislative responsibilities. Table 3 also shows a breakdown of the incremental costs associated with each individual piece of legislation, relative to the baseline. This shows that delivery of annual demography statistics are more costly to NSIs/Member States than delivery of migration statistics. However, costs related to census statistics are considerably higher for both NSIs and Eurostat than those related to the production of annual demography or migration statistics. As shown in Figure 9, the overall and per capita costs vary greatly depending on the census method. For example, the costs associated with a register-based method – becoming more and more common across Member States – are much lower than for a traditional or combined census method. This is substantiated by findings from the NSI survey, where 12 out of 31 respondents suggested that the use of administrative sources would help improve the cost and organisational efficiency with which statistics are produced. Generally, the key cost drivers for producing European statistics on population under the current legislative framework include the cost of staff, IT equipment and assuring data quality. Feedback from the NSI survey suggests that the main drivers of costs are staff and IT: 16 out of 30 respondents to the survey said these were the key cost drivers. Other drivers suggested by respondents included the quality of data, changing variables, and data collection. EI6 – What benefits do data users currently get from European statistics on population (i.e. baseline benefits)? → The existing legal framework improved key gaps from the 2000s. This led to various benefits that were identified and valued by all stakeholder groups, most notably improved comparability across Member States and better evidence for decision-making. This evaluation has shown that the existing legal framework succeeded in improving key policy gaps from the 2000s by improving data availability (RI1) and quality (SQ1-6). This helped to add significant value at EU level (EU1-5). According to the analysis by the contractor, this generated the key benefits set out in the bullet points below (see full list in Annex 2). ·Benefits to the general public: improved access to data and evidence, more accurate media reporting, increased awareness of population data, and greater engagement with the data-gathering process. In the longer term, the improved statistics will bring benefits associated with improved policymaking at the EU and Member-State level. ·Benefits to NSIs/Member States: increased access to higher quality and more timely data. Staff are more skilled in – and have greater ownership over – the data-collection process. NSIs and Member States are better able to collaborate with data owners and meet user needs. In the longer term, the improved statistics will bring economic and social benefits related to better understanding of population data and associated improvements in policymaking. ·Benefits to Eurostat: reductions in administrative burden associated with the coordination and interpretation of data provided voluntarily. Eurostat also benefits from an improved reputation and is better able to meet user needs and evolving policy needs. ·Benefits to the EU more widely: increased access to high-quality, timely, reliable and detailed statistics to feed into decision-making. This includes being better able to make informed policy decisions which in turn improves the EU’s reputation. ·Benefits to non-institutional data users: increased access to high-quality, comparable, reliable, timely and detailed statistics. This leads to non-institutional data users being better able to produce detailed research and comparative analyses across Member States. Given the nature of the types of benefits identified, it was not possible or appropriate to quantify these. However, according to feedback from the interviews conducted, NSI survey, OPC and documents reviewed, overall the main benefits are centred around the legislative framework’s ability to enable access to high quality data to support the development of evidence-based policy while reducing or minimising costs for data users. Figure 10 makes this point more clearly. It shows the views of OPC respondents on the various benefits brought by the intervention, with many respondents agreeing that the intervention helped comparability across Member States (77% agreed), better decision-making (68%) and enhanced quality of products and services (66%). These findings were corroborated by feedback from the NSI survey, where 20 out of the 33 respondents saw the availability and comparability of European data as the main benefit of the intervention. A further 7 respondents said that the main benefit of the intervention was that the needs of data users were now being met. Stakeholders who were interviewed also said they valued Eurostat’s role in providing guidance, press releases and news based on European data; checking and validating data; and providing assurances around the comparability and reliability of data. 5.5.Coherence – internal (CI) CI1 – To what extent do the legal bases cover statistical objectives through mandatory data collections?

→ The intervention initially managed to regulate many datasets serving policy needs, thus reducing users’ dependence on voluntary datasets. However, dependence on voluntary datasets increased once more due to the need for specific data addressing the COVID-19 pandemic from 2020 onwards. An EU framework that is designed from the outset to put current and evolving data needs on a regulatory base – at least to a proportionate extent and in the medium or long term – can be considered more internally coherent than one that does not achieve this. From that perspective, the Census Regulation was internally coherent by enabling delegated mechanisms to regulate the entire content (within the scope defined in the base act) before each census round. Similarly, the Demography Regulation managed to reduce the number of voluntary datasets by 35% from 2014 on (CI1.1), although it remained necessary to collect voluntary input data (regional or other breakdowns) for regional population projections and ad hoc policy requests. The renewed sharp increase in voluntary datasets in 2020 (up by 39% from 2019 in response to the needs created by the COVID-19 crisis) is not a problem in itself. Rather it is the lack of flexibility of the legal framework to regulate some of these datasets in the medium or long term that reduces internal coherence. CI2 – Is the current legal framework internally coherent? → The legal framework is generally internally coherent and the various legal acts operate well together in achieving their objectives. Some internal inconsistencies remain around the weak and non-harmonised provisions on population-base definitions across legal acts. According to the contractor’s analysis, the legal framework governing European population statistics is generally internally coherent overall. The various legal instruments covering population data collections also operate well together in achieving their general and specific objectives. However, certain issues have been identified with insufficiently consistent statistics, such as in the area of migration statistics or other statistics on population collected by Member States based on different definitions. These issues could be caused by inconsistencies in the existing legal framework. Eurostat currently undertakes several separate collections of population data with different periodicities and under different legal bases. Given that these statistics are currently based on several legal acts that were not designed jointly, definitions and disaggregations of data were not developed together. This has resulted in sub-optimal internal coherence. The analysis of the legal framework shows that there are no major legislative gaps, but rather a series of inconsistencies that could be improved under a further harmonised EU legal framework governing population statistics. Such inconsistencies mainly stem from legal provisions that are not internally fully coherent. These legal provisions grant Member States certain flexibility in defining their population base for different datasets. Figure 11 shows that, when keeping the flexibility to choose national population-base definitions, Member States accept a trade-off between negative impacts at EU level and the main positive impact of cutting production costs. Other significant reasons for maintaining national definitions given by most NSI survey respondents include constraints in available data sources (66% cited this as a reason), existing national legal frameworks (58%), and historical context (55%). One final inconsistency is that the current legal framework endorses a variety of data sources (including administrative sources) to be used for statistical production. However, the current legal framework lacks strong sectoral provisions to enable producers of official population statistics (mainly NSIs) to process relevant sources for statistical purposes. As a result, the reuse of administrative data is not fully embraced or operationalised for all relevant sources available across various Member States. Nevertheless, question EI4 (section 5.4) has established that gaps in practical cooperation with data owners are typically more relevant than a lack of enabling legal provisions. Moreover, according to an earlier Eurostat study (footnote 35), data-protection restrictions stipulated in legal acts governing specific data sources are another main factor that lead to conflicts with NSIs’ legal right to access these sources. This means that additional legal acts must often be passed to allow NSIs to access these specific data sources. 5.6. Coherence – external (CE) CE1 – To what extent are population statistics coherent with related or depending other European statistics? → The coherence of population statistics with related or depending other European statistics is satisfactory. The intervention led to improved coherence with managed-migration and national accounts data. An important element in the intervention’s external coherence is the level of statistical coherence of the data it produces with related statistics produced outside the intervention. Most relevant information overlaps in this respect are in the domains of: (i) asylum and managed-migration statistics; (ii) social surveys based on samples; (iii) national accounts statistics; and (iv) health statistics. Some findings from respective indicators are summarised below. ·CE1.1 – Coherence of data on immigrants (flows and stocks) with data on residence permits issued (from asylum and managed-migration statistics): despite a general rough correlation between these data, large discrepancies (up to >100%) and large variations between ESS members (smallest around 2%) are observed. The most significant reasons for these discrepancies were identified, and several NSIs are trying to quantify their difference components (accounting scheme, work in progress). ·CE1.2 – Censuses used as a sampling frame for other social surveys: the use of census results in this respect has decreased between the baseline and 2017 (post-intervention). Nevertheless, the census remains important for this purpose in about half of the ESS members. This reduced role of the census is likely a consequence of the general transition towards register-based production systems (two thirds of ESS members mainly use administrative sources for sampling frames). ·CE1.3 – Census and annual demographic statistics used for national accounts: census results continue to play a prominent role in the compilation of national accounts, especially housing data. Demography statistics are not used in national accounts – instead separate data are produced within the national accounts domain that lead to the question of coherence (next point). ·CE1.4 – Coherence of annual population counts with respective national accounts data: average total population during a reference year is an indicator in the ESOP dataset demo_gind, but is also produced independently in the national accounts dataset nama_10_pe. The average absolute deviation between these indicators across Member States is stable at below 0.2% for the entire period since 2005. The largest absolute values are consistently below 2% and there are no notable trends or effects from this intervention. Therefore, the data are largely but not fully coherent, which is to be expected due to methodological reasons. ·CE1.5 – coherence of statistics on annual deaths with health statistics on total deaths: the discrepancies between these statistics over the years of their availability (2011-2018) remain mostly around or below 1% for all Member States and other reporting countries. There are conceptual differences in the data collections and different data sources are used. There may also be different definitions. Systematic biases over the whole time series are visible at national levels. So far, Eurostat has not systematically analysed the specific national drivers for these biases in each Member State. CE2 – To what extent are EU concepts and definitions harmonised with international practices or recommendations? → The European Statistical System, coordinated by Eurostat, participates in international efforts to harmonise concepts and definitions of population statistics. The few notable gaps identified relate to the population-base definition. Eurostat contributes to – and actively promotes – international statistical cooperation. This creates an environment of common concepts and definitions that ultimately foster international harmonisation of official statistics. A key element of this framework is the set of recommendations on censuses of population and housing organised and updated every 10 years by the UNECE and endorsed by the Conference of European Statisticians (CES). These CES recommendations are applicable – and intended to be relevant – to the needs of the 56 member countries of the UNECE. They are broadly comparable with the global census principles and recommendations coordinated by the UN Statistical Division. Moreover, as a key stakeholder of the European Statistical System (ESS), Eurostat is ideally placed to provide support to non-EU countries and non-EU institutions that are looking to approximate their official statistics to EU and international statistical standards. The current population definition in the EU census legislation is based on the usual-residence concept with a residence rule that requires 12 months of residence. However, this current population definition allows a default to national concepts based on legal or registered residence without a time criterion (RI2.2), which is not formally in line with international recommendations that rely on a strict usual-residence concept. In practice, the national data supplied to the United Nations or the OECD generally use the same population definitions as those data supplied to Eurostat. This means that there will be coherence between the data for a particular country supplied to Eurostat, the United Nations or the OECD. However, there will be international inconsistencies with the population definition and its application by different countries.

CE3 – Is the current legal framework coherent with other EU policies and legislation, including the Charter on Fundamental Rights? → The legal framework is generally coherent with other EU policies and legislation, including the Charter on Fundamental Rights. The gap in data on equality is not related to incoherent legislation. According to the contractor’s analysis, the legal framework governing European population statistics is generally coherent with other EU policies and legislation, including the Charter on Fundamental Rights. The contractor’s analysis also concluded that the EU’s legal framework operates well together with other international legal instruments that also cover population statistics. In the OPC, 52% of respondents agreed with these conclusions, whereas only 15% disagreed (the remainder were neutral – 14%, or uncertain – 19%). No major legislative gaps were identified, but several stakeholders mentioned issues related to the collection of data on equality. For instance, the European Committee of Social Rights has identified a duty for national authorities to collect such data to inform policies. And at EU level, these data are repeatedly called for to support EU policies. However, during consultations with NSIs, some countries indicated that their national legislation does not allow them to collect statistics on equality data, such as data on ethnic groups and other types of data considered as sensitive. Nevertheless, Commission studies concluded that no Member State currently imposes an absolute prohibition on collecting data on ethnicity, sexual orientation or gender identity. Therefore, the data gap seems less the result of possible conflict between legal instruments and more the result of three things: (i) technical/statistical feasibility (especially with production systems largely based on administrative sources); (ii) cultural preconditions; and (iii) priority in certain Member States. One important synergy is the synergy between the ESOP framework and the framework for European statistics on persons and households based on samples (footnote 9). In 2017, Eurostat’s modernisation programme for social statistics suggested separate framework regulations for social sample surveys and for population and census statistics. However, the two frameworks are linked in many aspects. This is because the ESOP framework provides the population frames for data collections from samples (see CE1.2) as well as the necessary tools for benchmarking the data covered by samples. 5.7.EU added value (EU) EU1 – To what extent is statistical quality achieved at EU level? → The key elements of statistical quality adding value at EU level are completeness and comparability across Member States. The intervention has improved both of these quality dimensions significantly for the data that became regulated. As noted earlier, the main value added at EU level for statistical quality is in completeness and comparability. Improvements in these two areas can be measured quantitatively based on the published data, but also qualitatively based on the opinions of key stakeholder groups from the OPC. On completeness (EU1.1), the quantitative analysis is based on indicators SQ6.1 and SQ6.2 (Figure 16). This quantitative analysis illustrates how the common legal base (including provisions on data contents and deadlines for sending the data) has improved the EU-level completeness of regulated data from below 45% at the baseline (up to 2005) to essentially 100% post-intervention. However, the quantitative analysis also shows that voluntary data remain at a lower score of below 60% to this day. Most OPC respondents from most stakeholder groups find that the statistics are sufficiently complete. On comparability (EU1.2), the quantitative analysis is based on indicators SQ1.5 and SQ2.1-SQ2.3. This analysis shows that regulation again improved the situation where it was effective (SQ2.3), but that comparability gaps remain in voluntary data (SQ2.2) or where the regulation was not effective in harmonising concepts (especially regarding the population base – SQ1.5 and SQ2.1). Most OPC respondents across most of the stakeholder groups said that the statistics were sufficiently harmonised/comparable at EU level overall. However, most respondents across most stakeholder groups agreed only ‘somewhat’ that the legislation ensured comparability between Member States. There is therefore a positive opinion overall that the legislation ensures comparability, but most stakeholders are also aware of certain limitations. EU2 – To what extent did the intervention achieve methodological soundness at EU level (harmonisation of definitions and implementation, including the population base)? → The legal framework does not establish a strict common population base. This has led to a fragmented landscape of national definitions used by Member States. This in turn had led to reduced comparability and a risk of double counting at EU level. The legal definition of usual residence provided for in the three Regulations of the legal framework provides for some flexibility: where the circumstances for usual residence based on 12 months of actual presence cannot be established, the legal or registered residence can be used instead. The problem with these defaulting options is that they are not defined legally. In particular, there is no duration-of-residence criterion. This means that the conceptualisation and implementation is largely subject to national legal, administrative or policy contexts. This leads to a situation where there could be up to 27 factually different national definitions of the population base. Under Article 8 of the Demography Regulation, Member States were required to carry out feasibility studies on the use of the definition of ‘usual residence’ for population and vital events. In 2016, Eurostat analysed these feasibility studies, and reported its findings on the current implementation of the definition of population base across all Member States under the current EU legal framework. The results showed that Member States were indeed using different population-base definitions, including individual Member States using different population-base definitions for statistics under different legal acts. Figure 12 shows updated information from the targeted NSI survey illustrating the fragmented landscape of population-base definitions that remains to this day. Interestingly, the OPC results show that significant minorities or even majorities of respondents across all stakeholder groups (more than 42% in each group except for researchers and statistics producers) were not aware of this fragmented landscape. However, a majority across all groups – including statistics producers – agrees (often even ‘strongly’) that harmonisation is important. In-depth interviews conducted by the contractor during the case studies highlighted that NSIs consider their national definitions to be adapted to the national context. The interviews also show that NSIs consider the benefits of the current use of national definitions to centre around meeting current national requirements. Some NSIs asserted that this can lead to issues with not having entirely comparable statistics at EU level. These NSIs said this can be an issue for some data users that had a greater need for comparable population statistics (those for whom precision at granular level is required). The use of national definitions can also give rise to issues of double counting when people move between Member States, and this can cause discrepancies in European population statistics. Data users and international partners of Eurostat identified similar problems, for instance when counting people who migrate between Member States to study or work, or where people have second homes and spend parts of the year living in two or more places. EU3 – To what extent are the users satisfied? → Users are generally of the opinion that European population statistics are of high quality overall and add value compared to other sources of statistics. The overall level of appreciation has been high since at least 2009, during the implementation of this intervention. Eurostat regularly conducts standardised user satisfaction surveys (USS) that provide a certain level of breakdown by topic areas. These can be used to obtain a time series to benchmark the OPC results. Figure 13 shows USS results for 2009-2020 on overall quality of the relevant area ‘Population and social conditions’ (more specific results for population statistics are not available) and compares these results to the average over all areas. There is a continuous high rating (80% or more of users ranking it as ‘Very good/Good’) with almost no variation between 2009 and 2019, both in the relevant area and in the average. The small annual variations are very similar between the area-specific and average results, which suggests an effect mainly from variations in respondent groups between the years. In the OPC, a vast majority of respondents across all stakeholder groups (more than 75% in each group) agreed that the statistics are of high quality overall. This aligns with the results seen in the USS time series, when the categories ‘Very good/Good’ and ‘Adequate’ are added together. Another question in the OPC was whether the current EU legislation adds value compared to other sources (national and international). Here again, the vast majority of respondents from all stakeholder groups (70% or more in all groups) agreed with the statement. The same high level of agreement of 70% or more is also found across all stakeholder groups on the question of whether the legislation enables them to obtain statistics from a single source. These responses need to be contextualised by feedback from EU-level institutional users (Commission DGs), who continue to stress key gaps and shortcomings for EU policymaking (see RE1, Section 5.2) both in the targeted consultation and on other occasions (e.g. regular sectoral Eurostat hearings with main user DGs).

5.8.Statistical quality (SQ) Statistical quality dimensions evaluated in this section follow the ESS handbook for quality and metadata reporting (2021 re-edition), which is based on the European statistics Code of Practice (2017 revision) rooted in Articles 11 and 12 of Regulation (EC) No 223/2009 on European statistics (footnote 17). SQ1 – Coherence → The intervention has largely improved coherence and consistency between datasets, but gaps remain due to the lack of harmonisation in the population base. Within the scope of this evaluation, coherence refers to the compatibility of information across different datasets produced by the same entity (NSI or other) but potentially under different base acts (e.g. census vs annual population vs migrant population, etc.). This is in line with the ESS handbook conceptualisation, which compares data produced from different statistical processes. For instance, at the baseline, total population aggregates were notoriously inconsistent between annual demographic and migrant population datasets. Only the Demography Regulation links the population used at national level to that of the migrant stocks, so that these inconsistencies disappear from 2014 onwards (SQ1.3, Figure 14 left). Similarly, population stocks from 2011 are largely consistent with the census results of the same reference year for 25 ESS members (SQ1.4). However, the harmonisation gap for the population base under the current legal framework (question RI2) leads to coherence issues that have persisted since the intervention. For instance, according to the metadata, 18 ESS members nominally apply the strict usual-residence concept based on a twelve-month stay for all annual population-stock data at national level (2020, SQ1.1). However, 5 of these 18 report different population totals between the dataset for Council voting weights (where strict usual-residence rules are enforced) and other annual population-stock datasets in 2020. This suggests an incoherent implementation of the usual-residence concept at national level (SQ1.2). In other situations, different population bases are used for population stocks and migration flows, which leads to inconsistencies between stock differences and demographic changes (demographic balance) between reference years. For any reference year since the baseline, between 15 and 21 ESS members have reported inconsistent demographic balances (SQ1.5, Figure 14 right). The intervention did not improve this situation because the underlying harmonisation gap was not resolved from the outset. SQ2 – Comparability → Generally, comparability at EU level is high and this is acknowledged by users (EU2). However, the lack of harmonisation in the population base has a negative impact. According to the ESS handbook, comparability refers to data from nominally the same statistical processes across geographic regions and/or time spans. For this evalua