EUROPEAN COMMISSION Brussels,25.4.2018 SWD(2018) 145 final COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Proposal for a Directive of the European Parliament and of the Council on the re-use of public sector information {COM(2018) 234 final}



{SWD(2018) 129 final}



TABLE OF CONTENTS 1.Introduction 2.Background to the intervention 3.Method 4.Implementation / state of Play 5.Analysis and answers to the evaluation questions 6.Conclusions ANNEX 1 – EVALUATION QUESTIONS ANNEX 2 – GLOSSARY ANNEX 3 – LIST OF LITERATURE USED 1.Introduction Purpose of the evaluation This report provides the results of the evaluation under the Regulatory Fitness and Performance Programme (REFIT) of the Directive on the re-use of public sector information (PSI Directive). In line with the "Better Regulation" guidelines, the evaluation assesses the effectiveness, efficiency, relevance, coherence and EU added-value of the PSI Directive, and aims to identify opportunities to reduce regulatory costs and to simplify the existing legislation without negatively affecting the achievement of the underlying policy goals. In the May 2017 Mid-Term Review of the Digital Single Market Strategy (COM(2017) 228 final), the Commission announced that it would, "in spring 2018, based on an evaluation of existing legislation and subject to an Impact Assessment, prepare an initiative on accessibility and re-use of public and publicly funded data and further explore the issue of privately held data which are of public interest". With the review of the PSI Directive, the Commission can fulfil the periodic review obligation contained in Article 13.1 of the Directive, which stipulates that "the Commission shall carry out a review of the application of this Directive before 18 July 2018 and shall communicate the results of that review, together with any proposals for amendments to this Directive, to the European Parliament and the Council", and at the same time contribute to fulfilling the goals of the Digital Single Market Strategy in the field of the data economy.

Pursuant to this commitment, this REFIT evaluation has been carried out back-to-back to the Impact Assessment on policy options for the future of the PSI Directive. The conclusions of this evaluation have – where relevant – fed into that Impact Assessment. Scope of the evaluation The evaluation covers the overall functioning of the Directive, but has a special focus on the changes introduced in 2013. Although the overall functioning of the Directive was already evaluated once in 2009, it is believed that the evaluation of the 2013 changes cannot be performed in isolation and must be linked to the overall assessment of the effects of the Directive. National transposition measures are not part of the scope of this evaluation. This evaluation includes all EU Member States and covers the period starting from the transposition deadline of the Directive (July 2015) to January 2018. 2.Background to the intervention Situation prior to the intervention Public bodies produce, collect and hold a wealth of information and content, ranging from statistical, economic or environmental data to archival material, collections of books or works of art. Prior to the 1990s this information was difficult to access and usually solely available in paper formats. However, the digital revolution that started in the 1990s put the spotlight on the potential of public sector information (PSI). PSI (and data in general) soon started to be seen as important raw material that, if made available for re-use, could become a valuable economic asset and an important resource for economic growth and competitiveness. The focus on PSI resulted in a number of initiatives at the national, European and international level. However, with this renewed interest in PSI, public sector bodies started trading with it, which soon led to distortions of competition in the internal market. At the European level, the first step to address these issues was the development of a set of guidelines to promote access to information, transparency, and a level playing field already in 1989 , but it took another 10 years before the debate on open public data was properly launched. In 1998, the Commission published a Green Paper , which played a major role in raising the debate across Europe on the opportunities created by the re-use of PSI in digital format that went beyond the purpose for which it was originally collected. The Green Paper was followed in 2003 by the Public Sector Information (PSI) Directive (2003/98/EC). The Directive aimed to facilitate the re-use of PSI throughout the Union by harmonising the basic conditions for making PSI available to re-users, to foster the development of Community-wide products and services based on PSI, and to avoid distortions of competition. The Directive is based on Article 114 of the EU Treaty and builds on key pillars of the internal market: transparency and fair competition. The Directive succeeded in harmonising the basic legal framework for re-using PSI in the internal market in order to facilitate the re-use of PSI across the EU, where the Member States used to apply markedly different rules and practices, and in order to eliminate certain anti-competitive practices by public sector bodies. Nevertheless, despite this achievement, implementation of the Directive and progress in PSI re-use across the Member States was uneven and much of the potential of PSI remained untapped as many resources were locked in, difficult to find or made available on prohibitive terms, thereby preventing development of the national and EU PSI re-use markets. This potential had further grown because of technological advances: the so-called web 2.0 showed the potential opportunities related to mixing different data sets, and visualising them through so-called mash-ups. This was further reinforced by the sudden development of the mobile app market, kick-started by the launch of the iPhone ecosystem in 2007. Therefore, ten years after it came into force, Directive 2003/98/EC was amended by Directive 2013/37/EU in July 2013, with the aim to encourage Member States to make as much material held by public sector bodies available for re-use as possible. This intervention was part of a range of initiatives implemented in the framework of the Digital Agenda for Europe and the Europe 2020 strategy for smart, sustainable and inclusive growth. The 2013 revision addressed a series of problems, which were still present as barriers to re-use of public data across all the EU Member States: ·insufficient clarity and transparency; ·locked resources; ·excessive charging and lack of a level playing field; ·inconsistent approach across the Member States; ·insufficient enforcement of re-use provisions. To address the above concerns, the general objective of this intervention was to contribute to economic growth and job creation by improving the conditions for the exploitation of PSI and facilitating the further development of the internal PSI re-use market. Moreover, facilitating the re-use of PSI further was supposed to have a positive effect on the transparency, efficiency and accountability of governments and contribute to citizen empowerment. This general objective was in line with the original objectives of the 2003 intervention, which were to: 1)prevent distortions of competition on the EU market;

2)stimulate the digital content market for PSI-based products and services; 3)stimulate cross-border exploitation of PSI. Furthermore, for each of the specific objectives presented above, a number of operational objectives or ‘measures’ were identified, as illustrated in the graph below: On a more granular level, the 2013 intervention aimed to achieve the above objectives by addressing the following areas of concern: 1)Limited scope of application of the PSI Directive: the 2013 changes extended the scope of application to the cultural sector; 2)High charges for the re-use of PSI: in the 2013 version of the Directive the principle of charging the marginal cost was laid down (although there are a number of exceptions); 3)Limited re-use linked to formats in which PSI is made available: the 2013 changes included a recommendation that public sector bodies should, as far as possible, make PSI available in open and machine-readable formats. The core indicator of progress towards meeting the identified objectives was the correct transposition and application of the PSI Directive. Progress in the re-use of PSI and related policies across the EU was supposed to be measured in accordance with the following indicators: ·number of data portals and of available datasets; ·quality of the datasets evaluated according to a set of criteria including machine readability and the possibility of having user feedback; ·level of re-use (number of downloads of data, number of downloads of applications, turnover of companies); ·cross-border use of data; ·number of exceptions from the general rule on pricing, and their economic impact; ·degree of standardisation of licensing conditions, use of open licenses; ·level of activity of redress and enforcement mechanisms, including the degree of independence of regulatory bodies and effectiveness of their decisions; ·number and character of complaints and other reports from citizens and business to the Commission. In addition to the Directive, several support measures were put in place:

·Guidelines on recommended standard licences, datasets and charging for the re-use of documents adopted in 2014 (2014/C 240/01) ; ·creation of an open data portal for EU documents and a European data portal infrastructure federating existing open data portals, including support services; ·funding of research and innovation projects. 3.Method The evaluation took place between March 2017 and January 2018 and drew from the following main data sources: ·Study to support the review of the PSI Directive The review process was supported by a study, conducted by a consortium, on the functioning of the PSI Directive. The aim of the study was to assist the Commission in evaluating the existing legal and policy framework applicable to data access and re-use (assessing the role which the PSI Directive has played in promoting PSI re-use across Europe), and checking whether it could be improved to address some of the weaknesses identified and/or new issues which have emerged since the last revision of the Directive (in particular by assessing the expected impacts of a number of policy options/combinations thereof). The study relied on a combination of sources and methods, including strategic interviews, desk research, interviews with stakeholders at the EU and national levels, workshops with public sector practitioners and academics, as well as with PSI re-users and data economy players, three online surveys with public authorities, including public cultural institutions, education and research bodies, as well as the re-user community, and analysis of the public online consultation launched by the Commission. The original data collection focused on a representative sample of 10 EU Member States: Estonia, France, Germany, Greece, Ireland, Italy, Netherlands, Poland, Slovenia, and Sweden. ·Open Data Maturity in Europe - 2017 Report Every year, Open Data maturity across Europe is measured for the Commission through a series of indicators. The factors taken into account for measuring Open Data maturity are aligned with the provisions of the PSI Directive and the practical measures necessary to make data better findable and usable (see the indicators listed under section 2). The measurements cover the level of development of national policies promoting Open Data, an assessment of the features made available on national data portals as well as the expected impact of Open Data. This work is carried out by a consortium contracted by the Commission and responsible for the European Data Portal measures. ·Input from Member States The Commission has analysed the latest available Member States reports as part of its monitoring of the PSI Directive pursuant to Article 13(2) of the Directive. Additional input from Member States was received at a meeting of the Member State Expert Group on PSI on 15 November 2017. ·Stakeholder consultations Broad stakeholder consultations were organised, covering the following actions: 1.Inception Impact Assessment An Inception Impact Assessment was published on the dedicated Better Regulation webpage and it was available for feedback for 4 weeks (18 September 2017 - 16 October 2017). Seven stakeholders sent replies. 2.Public online consultation A public online consultation was published on the dedicated Consultations webpage and it was available for feedback for 12 weeks (19 September 2017 - 16 December 2017). All interested parties, including governments, public sector content holders and users, commercial and non-commercial re-users, experts and academics as well as citizens were invited to contribute. The online questionnaire covered both the evaluation of the current Directive implementation and the problems, objectives and possible options for the future. Respondents also had the opportunity to upload a document, such as a position paper. With several targeted actions, the Commission made stakeholders aware of the public online consultation and called on them to participate. Feedback was received from 273 stakeholders. A total of 63 contributions received until the end of the consultation process, end of January 2018, were taken into account.

3.Targeted stakeholder events As part of the evaluation and impact assessment process, the following stakeholder events were organised to address particular issues and/or target specific stakeholders: ·26 June 2017: workshop on access for public bodies to privately-held data of public interest; ·14 December 2017: workshop on open research data; ·18 January 2018: workshop with PSI holders and re-users; ·19 January 2018: public hearing (open to all) on the review of the Directive on the re-use of public sector information, with 98 attendees and 19 speakers; ·23 January 2018: high-level roundtable on opening up data in the transport and utilities sectors. 4.Ad hoc meetings Several ad hoc meetings have taken place with stakeholders' representatives, such as E3PO, the Union internationale des Transports Publics, the German Verband Deutscher Verkehrsunternehmen and Compass Gruppe, and the French Union des Transports Publics and Institut de la Gestion Déléguée. ·Literature review Literature review, including several studies on the impact of open data on economic growth and innovation, impacts on society, the impact of lowering charges on re-use, and others. Limitations – robustness of findings The data collection and analysis carried out has a number of intrinsic limitations, whose impact was mitigated to a maximum possible extent: ·Numerous sources that cover the data economy in general, the value of open data, issues related to PSI, and other similar topics have been identified and taken into account. There are differences between the studies in terms of their objectives and scope, their specific approaches towards quantitative and qualitative data, as well as the types of specific estimates they contain. An important difference between the studies is their focus. Whereas some concern the data economy in general, others concern open data or PSI itself. ·The study to support the review of the PSI Directive used a representative sample of 10 EU Member States, selected on the basis of geographic balance and the score in Open Data Maturity index. This naturally limits accordingly the findings on an EU-wide level for all 28 Member States. ·The evaluation takes into account the inherent limitations of the findings of public consultations. Firstly, as in all surveys, the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who have a stake in this domain. Secondly, stakeholders' views convey an individual rather than a holistic perspective. Based on the elements above, this evaluation has been carried out on the basis of the best available data.

Pursuant to the Commission Better Regulation Framework, the Directive has been subject to an overall assessment, as well as to a more specific evaluation process against the following five criteria: effectiveness, efficiency, relevance, coherence and EU added-value. The evaluation questions for each criterion can be found in Annex 1. 4.Implementation / state of Play Following the last revision, the implementation of the PSI Directive by the Member States is monitored by the European Commission on the basis of Article 2 of the amending Directive. It provides that "by 18 July 2015, Member States shall adopt and publish the laws, regulations and administrative provisions necessary to comply with this Directive. They shall immediately inform the Commission thereof" and "shall apply those measures from 18 July 2015." At that time, full transposition was notified to the Commission by 11 Member States. For this reason, the Commission started in September 2015 formal infringement proceedings against 17 Member States that had not yet notified the relevant national implementation measures. The last Member State notified transposition on 15 September 2017. The table below contains an overview of the way in which the Member States have transposed the 2013 revised Directive. Type of action taken Member States Adoption of specific measures providing for re-use of public sector information 13 Member States: ·Belgium ·Cyprus ·Germany ·Greece ·Spain ·Hungary ·Ireland ·Italy ·Luxembourg ·Malta ·Romania ·Sweden ·United Kingdom Combination of new measures specifically dealing with re-use and existing legislation 3 Member States: ·Austria ·Denmark ·Slovenia Adaptation of the legislative framework for access to documents to include re-use of public sector information 12 Member States: ·Bulgaria ·Czech Republic ·Estonia ·Finland ·France ·Croatia ·Latvia ·Lithuania ·Netherlands ·Poland ·Portugal ·Slovakia The latest Member State reports confirm that PSI is generally made available through national data portals. They mostly harvest metadata and datasets from institutional and local portals, but in a few cases data are uploaded directly on the national portal as the exclusive point of access. The national data portals contain asset lists of metadata and most countries have defined their categories of data based on the G8 Open Data Charter priority domains. Most countries have guidelines for the publication of open data attached to their national data portal, together with templates for the datasets and metadata. As regards the redress practices, Member States reports reveal that two different models are being used. The first relies on a designated, independent appellate body to handle requests for redress and can be found in Croatia, Cyprus, Ireland and the UK. The second, most common redress model relies on the administrative law of a particular Member State. As the internal structures of each Member State differ to varying degrees, so do the appeal avenues. At the EU level, over the last 5 years, PSI related issues have been the subject of 9 complaints to the Commission. Finally, it is important to point out that some Member States are moving faster on PSI re-use and are exploring new areas, while others limit themselves to the implementation of the Directive. For example, with the Loi Pour une République numérique France has started addressing the issue of data generated as a result of activities carried out by public and private entities carrying out public tasks in the transport and utilities sectors. Finland has enacted legislation which aims to enable the data economy to benefit from new sources of data. These efforts, while in line with the overall objectives of the PSI Directive, are not coordinated and risk undermining the level playing field for commercial re-users, as well as the development of cross-border applications in the EU. 5.Analysis and answers to the evaluation questions Overall the available evidence indicates that the PSI Directive works well and successfully contributes to the achievement of its main policy objectives, which are to stimulate the digital content market for PSI-based products and services, to stimulate cross-border exploitation of PSI and to prevent distortions of competition on the EU market. Since the adoption of the last version of the Directive in 2013, the Digital Single Market and the European economy in general have seen fast development in terms of data. For instance, the value of the European data economy has grown from 285 billion EUR in 2015, representing over 1.94% of the EU GDP, to 300 billion EUR in 2016. Moreover, the number of data companies increased from around 129,000 in 2013 to 134,000 in 2016. PSI re-use is particularly relevant for the creation of SMEs. For example, the project ODINE funded by the EU under the H2020 programme, an incubator for start-ups using open data, found a positive correlation between the maturity of the national open data policy and the number of ODINE successful applicants from that specific country. This exemplifies how a rich open data environment provides favourable conditions for innovators in this field. One third of ODINE members state that they would not exist without open data. This suggests that most of these businesses are likely to have started solely with the intention to exploit the value of open data or have since founding become so reliant on open data that they would no longer be able to exist without it. Another third said they would struggle to exist without open data which indicates a significant reliance on this resource. Open Data for a biking start-up Bike Citizens, an Austrian start-up, was established in 2011 with the aim of sharing valuable knowledge about cycling in urban areas. Having started with a simple bike navigation app and a smartphone mount, the company nowadays focuses on cycling promotion, app technology and data analysis for cities. Its key service, the Bike Citizens app, uses free geodata from OpenStreetMap, which is populated with Open Data from national mapping agencies and other public sector bodies (e.g. Direction Générale des Impôts in France, Surveying and Mapping Authority in Slovenia etc.), in order to generate different kinds of maps, including world maps. The Open Data sets allow to identify high-traffic streets and mobility patterns of everyday cyclists that are represented in heat maps. Using Open Data, the app offers the best route in 300 cities all over Europe. The company employs 25 people and has further expansion plans for Europe and the US. The better availability of PSI across the European Union is reflected in the Open Data Maturity Index 2017. This index shows that nowadays the majority of the EU Member States qualify as open data fast trackers or even trend-setters, which was not the case back in 2015, as the two figures below show.

Open Data Maturity of EU Member States in 2015 (Open Data Maturity Index) Open Data Maturity of EU Member States in 2017 (Open Data Maturity Index) The assessment of the level of Open Data Maturity for each European country is measured by two main categories. Open Data Readiness examines existing policy, licensing norms, the impact of Open Data, the usage of these data and the level of coordination at a national level. Portal Maturity focuses on the characteristics of each Member State’s national data portal(s), such as their usability, the reusability of the data and the spread across domains. Within each of these rubrics, a range of Key Performance Indicators (KPIs) are assessed. The following illustration shows that the past three years have seen improvements across all measures. Source: European Commission on the basis of the European Data Portal Dashboard. In 2015, the average EU-score for Open Data Readiness was a paltry 47%. In just two years this has risen sharply to 72%. As regards the Portal Maturity dimension, the numbers have risen from an average of 32% in 2015 to 76% in 2017. This general trend over the last few years clearly indicates that overall, in the broader EU Open Data policy, the PSI Directive has proven its usefulness by contributing to making more and more PSI available for re-use. EFFECTIVENESS Based on available evidence, it appears that the Directive has been largely effective in meeting the objectives specified in Section 2, although there are areas that require further work. The analysis below is based on the comparison between the objectives and the extent to which they have been achieved after the implementation of the provisions. Preventing distortions of competition on the EU market The PSI Directive has been very effective in ensuring that competition is not distorted, although there is still ground for improvement as regards specific issues. In terms of overall functioning of the PSI market, stakeholders agree on the positive effect the PSI Directive has had in harmonising practices and providing stability for data providers and re-users. It has helped to lay down a level playing field for all stakeholders and clarified rules and approaches for both data providers and re-users. An example of such a positive impact can be identified with respect to the use of “exclusive agreements” for re-use. The positive impact of the Directive in this respect is twofold: on the one hand, exclusive agreements were disincentivised through the establishment of a number of strict conditions for their validity and, on the other hand, rules were clarified for all public and private partners recurring to exclusive agreements. The result has been a significant decrease in the number of exclusive agreements for the re-use of data. One Member State took the Directive as an occasion to go further and banned exclusive agreements entirely. Indeed, in this country any exclusive agreement is automatically void. If the rules on exclusive agreements helped in reaching the objective of ensuring fair, proportionate and non-discriminatory conditions for PSI re-use, some challenges related to these practices still remain. The results of the online public consultation indicate that many respondents are not sure about the current situation. Only 38% (73 of 193 respondents) agree that exclusive agreements between public sector bodies and third parties are used only exceptionally and are strictly limited to the cases mentioned in the Directive (e.g. necessary for the provision of the public service). This shows that despite significant progress, some further actions may be needed to curb the practice of exclusive agreements. Also, rather than formal exclusivity agreements, some stakeholders have raised the issue of PSI data lock-in as regards certain datasets. Cases of PSI data lock-in might happen in a number of different situations, but what they all have in common is the fact that there is no formal willingness on the side of the public authority to establish a formal exclusivity agreement between parties.

Stimulating the digital content market for PSI-based products and services The PSI Directive has significantly contributed to maximizing the benefits of the PSI market in Europe, both in terms of the development of new services and the creation of new jobs, and there is ample evidence to substantiate this claim. In the replies to the online public consultation 81% respondents (157 of 194) agree that more data held by public sector bodies, including cultural heritage institutions, has become available for re-use, and as many as 73% (140 of 193 respondents) agree that PSI is increasingly becoming a source of innovative services and products. More than half of the respondents (54%) also agree that PSI has become more affordable, including for start-ups and SMEs. On a more granular level, the effectiveness of the PSI Directive with respect to the objective mentioned above should be measured both in terms of impact on the supply side (availability of more PSI for the data market) and demand side (number of services and products developed as a consequence of the availability of data). Regarding the supply side there has been a constant process of monitoring the open data maturity and readiness of the EU Member States coming from different initiatives: the European Data Portal, the Open Knowledge Foundation, and the Global Open Data Index etc. The data collected as part of this research suggest that the availability of public sector data in Europe has increasingly improved over time together with the quality of the open data initiatives. Next to the improved Open Data Maturity in the EU28, the number of datasets available on the local, regional, national and European portals have been constantly increasing over time. For instance, the statistics for both the European Data Portal (giving direct access to datasets from all the Member States at different levels of government) and Europeana (giving access to information from cultural institutions) showed a steady increase in the supply of PSI. Similarly, statistics from open data portals at the local, regional and national level confirm this trend. Therefore, as all this evidence points out, on the supply side of PSI there has been a constant improvement over time in terms of the number of datasets available and open data maturity of European countries. Although it cannot be taken for granted that this trend is the sole result of the PSI Directive, these data suggest that the Directive has definitely contributed to improving the supply market side of PSI in Europe and to making Europe one of the most advanced open data economies in the world. Measuring the demand side and especially the number of products and services created as a result of this availability of data is more difficult. Nevertheless, in the below graph, the estimated direct, indirect, and total PSI economic value are provided. The graph depicts the development of the direct, indirect, and total economic value of PSI in the 28 EU Member States between 2010 and 2017. In 2010, the total economic value of PSI is expected to have been around 140 billion EUR. It is considered that this value has increased in linear fashion to roughly 220 billion EUR in 2017. This is an increase of 80 billion EUR, i.e. 57% compared to 2010.

Direct, indirect and total economic value of PSI (EU28, 2010-2017) Source: Impact Assessment Support Study, Deloitte, 2018. As regards the number of jobs created thanks to PSI re-use, the total number of persons employed in the EU28 has increased slowly but steadily during the last years according to Eurostat. The number of persons employed in areas of knowledge-intensive activities has developed similarly. Thus, it can be inferred that the number of citizens employed in jobs relating to PSI has increased similarly from 2010 to 2017, albeit at a slightly higher growth rate. It is estimated that around 20,000 persons are employed in jobs directly relating to PSI today. The number of persons indirectly employed depending on PSI is, however, approximately 3.5 times higher. Bulgarian start-up using Open Data to provide services for cargo companies Transmetrics solution provides cargo companies with an accurate demand forecast and network optimisation based on Big Data and predictive analytics. This limits the empty space within cargo vehicles. Transmetrics uses various types of Open Data from public sector bodies for their analyses, e.g. infrastructural data, traffic information, road conditions, customs declarations. Since 2013 Transmetrics has grown from 4 co-founders to a team of 19 highly skilled part- and full-time employees, received many international awards, including Forbes Business Awards, and signed the contract with its first official customer in 2015. Currently, the team is working with several other world-known cargo companies. Stimulating cross-border exploitation of PSI The PSI Directive has also played a part in facilitating cross border re-use of data, although progress in this area could still be improved. This is shown very clearly in the replies to the online public consultation, in which only 34 % (66 of 192 respondents) feel that PSI circulates freely across the EU and that cross-border applications based on such information are easy to implement. However, data on the percentage of traffic to any given national open data portal that comes from outside the specific Member States, available on the European Data Portal, is more positive. Some Member States, in particular the smaller ones such as Estonia, Ireland and Luxembourg are doing well. Medium sized countries such as Austria and Belgium are also performing strongly. According to the available figures, only some larger Member States seem to lag behind in this respect, e.g. Germany, Italy or the UK. Another interesting aspect that can be analysed in this context is the general reusability of data within and across borders. As measured by a recent report , it seems to be constantly improving. The improvements in the reusability scores are driven by increased adoption of Application Programming Interfaces (APIs) and the improving availability of machine-readable datasets. According to other international sources , while 98% of businesses seem to use primarily data deriving from national sources, about 48% of businesses also use data from international sources. At the same time, the use of information obtained from international sources has risen over the past years. In some cases, the use of international sources is due to the lack of availability of national data that, nonetheless, is available through international organisations. It is to be noted that the European Commission seems to be the principal international community source for many of the companies across Europe when it comes to open data. 2013 changes to the PSI Directive Following the 2013 extension of the scope of the PSI Directive to cultural data, what emerged in terms of effectiveness of the Directive in this specific field is that it is early for a full-fledged assessment of the impact of this change in the Directive. Some Member States are still fine-tuning the national rules (e.g. rules on charging) and there is limited awareness of this topic on the ground, both in cultural institutions and among re-users. Also, the Directive entered into force in a context in which other initiatives on digitalisation and promotion of re-use of cultural data were already established. Distinguishing the effects of these different instruments and weighting their relative impact is therefore difficult. Finally, despite a clearer legal framework in terms of re-use of cultural data, a number of barriers remain for the implementation of the PSI Directive, especially in the domains of intellectual property rights, heterogeneity of licence conditions, variable quality of meta-data and charging. Nevertheless, there are signs indicating the impact of this change. For example, a survey of cultural institutions in Poland found that in the year after the transposition of the Directive there were 616 requests of re-use to these institutions, although the majority were addressed to only one institution, the national museum in Warsaw.

The change in the provisions concerning charging for PSI re-use was a key modification brought into the Directive in 2013. Article 6 of the revised PSI Directive establishes the principle that “when charges are made for the re-use of documents, these shall be limited to the marginal costs incurred for their reproduction, provision and dissemination” . As mentioned in the literature “marginal costing is an important principle, as in the case of digital material it would normally mean no charges apply”. The changes to charging provisions were driven by the objective of lowering barriers for re-users of data (and especially start-ups, SMEs and citizens who might be more sensitive to costs) and increasing the exploitation of PSI overall. There is some evidence on the extent to which the changes in the charging practices have been effective and have achieved the abovementioned objectives, although it is limited since the transposition of the Directive is recent. So far it appears that although a limited number of public sector bodies might have been concerned by the new charging provision, this change had the potential to benefit a high number of re-users, and in particular start-ups and SMEs. In the studies where specific types of data were analysed (e.g. meteorological, geographic and hydrographical data), it emerged that there is high price elasticity in these domains, that is to say that re-users are very sensitive and reactive to price. Concerning the data format provisions, the PSI Directive introduced in 2013 a more explicit definition of "electronic means" by adding and defining the terms "machine-readable format", "open format", "formal open standard" and "metadata". Thus, paragraph 1 of Article 5 recommends now that "public sector bodies shall make their documents available in any pre-existing format or language, and, where possible and appropriate, in open and machine-readable format together with their metadata. Both the format and the metadata should, in so far as possible, comply with formal open standards." As regards effectiveness of this change, there have been some marked improvements in machine-readability of datasets. In 2015, only 3 Member States provided more than 90% machine readable datasets to the European Data Portal. By 2017, this figure has risen to include 18 Member States. Currently 66% of datasets are being published in machine readable formats, but still a relatively high level of data publishing is made in formats that do not support machine readability and continue to be published in proprietary formats (e.g. .txt, .pdf, .xls). With the growing importance of dynamic data, the insufficient use of APIs is regularly recognized as one of the main barriers for data re-use. This was for instance emphasized in the Commission's 2017 public consultation on "Building a European data economy initiative" where the summary report noted that 68% of respondents clearly support an increased use of APIs. This problem was not dealt with explicitly by the 2013 changes to the directive, although it was addressed in the guidelines. Some open data professionals also warn that there is still a lot to be done for non-dynamic datasets, in particular key datasets, where the lack of machine-readability is blocking or greatly limiting the re-use of data. An additional area of improvement would be to provide high value datasets published with common more harmonised data models in the same level of granularity or scale in each Member State. The assessment of the findings available suggests that the PSI Directive has been overall effective in achieving its objectives. The analysed evidence indicates that the Directive has ensured fairer competition in the Digital Single Market, enabled the creation of new jobs and services and, to a lesser extent, enhanced an effective cross-border use of PSI by businesses. Data also suggests that the Directive has not yet reached its full potential and that a few (and sometimes new) challenges exist, such as PSI data lock-in. Moreover, although only a limited period of time has passed since the implementation deadline, early signs clearly indicate that the 2013 changes have been effective, particularly as regards charging and data formats. EFFICIENCY The analysis of the findings on the efficiency of the Directive points to the conclusion that there is a positive balance between the costs and its benefits, and therefore it can be seen as efficient. The overall benefits brought by the PSI Directive fall into two categories: economic and societal. As for the economic benefits, as mentioned earlier, the European economy has seen fast development in terms of data since the adoption of the last version of the Directive in 2013: the value of the European data economy has grown from 285 billion EUR in 2015, representing over 1.94% of the EU GDP, to 300 billion EUR in 2016. Moreover, the number of data companies increased from around 129,000 in 2013 to 134,000 in 2016. The economic benefits also materialise in terms of a growing Open Data market size, indicating the total sales volume based on Open Data. Existing recent macroeconomic studies all provide ambitious forecasts about the value of Open Data, estimating large gains as a result of Open Data. Zooming in from an overall view of the European Open Data economy into subsets of the Open Data economy, clear benefits are also found in terms of market size and added value. On a national level, studies have been recently undertaken to demonstrate the impact of Open Data. In Denmark for instance, research into the impact of opening up address data showed that both the direct and indirect benefits added up to a total gain of EUR 63 million in the 2005-2009 period. The most recent study conducted in Denmark undertaken in this regard estimates the socio-economic value of the open geodata at 3.5 billion DKK (460 million EUR) in 2016. In the Netherlands, the impact of opening up the basis registry topography was estimated at 11.5-14.5 million EUR in 2013 and 13.6 million EUR in 2015. In Spain, the overall financial turnover of the 'info-mediary sector' (composed of companies that create value added services thanks to the parallel re-use of both public and private sector data) based on 2016 data was estimated to range between 1.55 billion to 1.75 billion EUR, whereas the turnover derived directly from info-mediary activity would be somewhere between 600 and 750 million EUR. A recent Finnish study showed that the revenue of ICT companies utilising Open Data grew in 2012–2014 on average by over 17% more than the revenue of companies in the same sector that did not utilise data in their innovations. Opening up more PSI has also brought numerous societal benefits for the EU. Whether it is in the area of public transport, energy consumption or healthcare, citizens can now access a range of applications based on PSI that can make their daily lives easier. Numerous use cases in Member States show concrete examples in many fields. A French website for doctors, pharmacists and patients Medicatio is a website that seeks to publish data on all available drugs in France, and in doing so it aims to improve medical information and dialogue among doctors, pharmacists and patients, as well as addressing the growing problem linked to overconsumption of drugs. It captures Open Data from multiple French ministries and agencies, and it has already won several prizes. Besides the benefits, the PSI Directive has also brought a number of costs to stakeholders. As the obligations imposed by the Directive only apply to public sector bodies at the Member States level, these stakeholders can be considered as those bearing the vast majority of costs related to the Directive. Other stakeholders, such as businesses and re-users do not bear particular costs in this respect. One could identify different categories of regulatory costs related to the implementation of the PSI Directive:

·Direct costs (compliance and administrative costs): Compliance costs encompass investments and expenses needed to comply with substantive obligations contained in the Directive (this also includes technical costs needed to make the data technically reusable, such cleaning up of datasets, maintenance of portals, API costs, software costs etc.). Administrative costs are the costs borne by public sector bodies as a result of administrative activities performed to comply with the obligations (e.g. time and resources spent in figuring out licensing and license related costs, personnel training, communication activities, developing guidelines etc.). ·Enforcement costs. These costs are associated with activities linked to the implementation of the Directive, such as information and monitoring, complaint handling and adjudication. The figure below visualises the costs of opening up PSI for all affected public sector bodies in comparison with the overall government revenue and expenditure. It can be seen that the estimated overall costs to open up PSI are only a minor fraction of the total government revenue and expenditure. Although an increasing number of public authorities open up their data, it is believed that the total costs of opening up PSI (i.e. not the average costs per institution) have decreased as a result of the introduction of the 2013 revisions of the Directive, e.g. due to efficiency gains at the individual institutional level. Costs of opening up PSI for all affected public sector bodies

compared to total government revenue and expenditure (EU28, 2010-2017) Source: Impact Assessment Support Study, Deloitte, 2018. At the same time as governments open up their data, they can cut costs in other parts of government activity through the use of PSI. The use of PSI is expected to have contributed to a saving of around 20 billion EUR in 2017 alone. In addition to cost savings, public administrations are also believed to have benefited from increased revenue / taxation of PSI-based goods and services. The additional government revenue from PSI today is around 6 billion EUR per year (i.e. approx. 0.1% of total government revenue). In the following figure, the annual economic benefits of opening PSI are compared with the overall annual costs of opening PSI. The economic benefits of opening PSI are around 240 billion EUR today, whereas the costs associated with opening PSI are around 9 billion EUR. It is believed that the 2013 revision of the Directive has triggered a decrease in overall costs of making PSI available. It can be concluded that the benefit-cost ratio is roughly 26:1 today, meaning that the benefits of PSI today are 26 times greater than its costs. Considering only the direct economic value of PSI, the benefit-cost ratio is approximately 5:1 today. Costs and benefits of opening up PSI for different types of public sector bodies (EU28, 2010-2030) Source: Impact Assessment Support Study, Deloitte, 2018. Respondents to the online public consultation also confirm that that the cost-benefit analysis of the PSI Directive is overall fairly positive and that the latter is therefore considered efficient. Indeed, 66% (128 of 193 respondents) agree that the costs borne by the public sector bodies in implementing the Directive (e.g. adapting IT infrastructure, lower income from charges) are offset by socioeconomic benefits of re-using data (e.g. creation of new digital applications and products, increased transparency). As many as 72% (138 of 193 respondents) agree that compliance with the Directive requires better data management processes of public institutions which leads to cost savings and increased operational efficiency. This finding has also been confirmed by the Danish administration in regards to its Basic Data Programme. It is expected that once the initiative has been completely phased in, savings for the public sector in Denmark will have reached about 250 million DKK (33 million EUR) per year as a result of lower administration costs. However, as regards the redress mechanisms, as many as 44% of respondents to the online public consultation agree that in case a request for re-use is rejected and an applicant decides to appeal against the decision of a public sector body, the redress procedure at the national level is slow and inefficient, and it can be very costly. It should be noted that the 2011 Commission proposal for revising the Directive addressed this issue, but that the relevant provision was significantly watered down in the interinstitutional process. 2013 changes to the PSI Directive The evidence indicates that the changes in the rules concerning charging and data format have been efficient. There is consensus among stakeholders (both from re-users and public sector bodies) that charging at marginal costs or below is efficient when looking at the balance between the overall benefits for the society and the related costs. The machinery behind a charging system (e.g. accounting, marketing) is costly and a large part of the income of public sector bodies charging beyond marginal costs comes from other public sector bodies. When trying to look more closely at efficiency gains from the 2013 data format provisions, it is difficult to provide an average breakdown of costs as both the existing data infrastructures already in place, and the technical choices will have an important impact on the final figure. The issue of interoperability is becoming more and more relevant to address, in particular as more high value datasets are made available online. This has been evidenced in several recent communications such as those from the Open Data Institute and Open Knowledge International , which consider it to be one of the main issues to tackle in order to progress towards a higher impact from open data. In this context it is interesting to recall again the Danish Basic Data Programme, which aims to establish a system of shared key registers through the implementation of the "once only" principle that comes with interoperability standards and APIs for data exchange. It has been estimated that the total impact of the Danish Basic Data Programme, if applied to all Member States, would amount to more than 5 billion EUR (net revenues for both public and private sectors), once fully deployed. The very high estimated impact for the whole EU is also due to the fact that those key registers are made freely accessible to users (open data) in high quality and with the right data formats and means of distribution (those required by the users).

As regards the extension of the scope of the Directive to cultural data, it is too early to draw conclusions on the efficiency of the measure, given the low awareness among cultural institutions and re-users of the rights and obligations under the Directive. The costs of making data reusable constitute, however, only a small percentage of the overall costs borne by cultural institutions when they decide to make data available by initiating digitisation of their collections, as confirmed by data collected in Poland in the framework of digitalisation of cultural heritage. Indeed, the costs for digitisation of cultural heritage are widely considered to be very high. Administrative burden There are no indications that the PSI Directive creates a disproportionate administrative burden on public authorities. 57% of respondents in the public online consultation find that the provisions of the PSI Directive are easy to understand and implement by public sector bodies and re-users alike. However, some stakeholders have indicated that understanding the interplay between the PSI Directive and other related legal instruments (e.g. the Database Directive) requires a high level of expertise, and in this context clarification of this interplay would help in reducing the administrative burden. Apart from the issue mentioned above, the burden mainly concerns public sector bodies and consists of the costs of coordination, service implementation relating to processing requests to make data available for re-use and compliance with transparency requirements. Despite these requirements, the burden does not appear to be excessive. For example, according to a Dutch study commissioned by the Association of Dutch municipalities (VNG), the implementation costs of the 2015 national law transposing the PSI Directive (Wet hergebruik overheidsinformatie – Who) do not exceed 2000 EUR per municipality per year. On a more global level, there are also monitoring and reporting obligations under the Directive, but they do not appear to be excessive either. According to feedback from Member States, the elaboration of the most complete report required around 100 man/days, with shorter and more streamlined reports requiring much less time and effort. Another issue raised in the exchanges with Member States is the need to be able to deal efficiently with recurrent unjustified requests from the same requester. More generally, it has also been pointed out that the overall thrust of the Directive is favorable towards reducing administrative burden, because it encourages public sector bodies to proactively make as much PSI available online for re-use as possible, ideally in an automated way. Following this principle means that public sector bodies would be faced with fewer requests and fewer complaints, and the administrative burden and costs related to this would progressively decrease as more and more PSI for re-use becomes available online. The example of the Danish Basic Data Programme mentioned above indicates the financial benefits of this pro-active approach, which has the additional advantage of lowering the cost for re-users in dealing with the administration. Finally, the Commission's guidelines of 2014 have helped Member States with practical guidance that can further help public sector bodies in reducing their administrative burden by optimising the conditions for the re-use of PSI. The purpose of the guidelines is to provide non-binding guidance on the best practices concerning the use of available open standard licences, the elements to include in custom-made licences, datasets to be published as priority, the ways to make them more readily re-usable, the application of the marginal cost rule and the cost elements that can be taken into account for cost-recovery charging. A number of stakeholders have indicated that the guidelines have helped them save time and effort that would otherwise have been required to address these issues on their own. The analysis of the findings on the efficiency of the Directive points to the conclusion that there is an overall positive balance between the costs and the (quantitative and qualitative) benefits of this legislation. For the efficiency of the 2013 extension to the cultural institutions, it is too early to draw conclusions. RELEVANCE Based on the evidence available, the PSI Directive is still very relevant today because it responds to a large extent to stakeholders’ needs, still fulfills its policy objectives, and addresses the right issues to facilitate re-use. Through its provisions on charging, non-discrimination, licences, requests for re-use, transparency and technical issues, it addresses the key requirements for successful PSI re-use, in particular that the information is: ·known to the re-user; ·attainable for the re-user; ·can be used for the intended purpose of the re-user. In terms of more specific requirements, it has long been acknowledged that PSI re-users need: ·clear, transparent and harmonised rules for access and re-use across all Member States to avoid uncertainty linked to different approaches and regimes at the national level; ·quality metadata; ·the availability of more datasets; ·lowering of charging for re-use of data to stimulate development of new products and services; and ·data to be provided in a standard, machine readable and re-usable format. The majority of stakeholders interviewed agreed that the PSI Directive responded to these needs (although sometimes not to the full extent – e.g. research data are currently not included in the scope of the Directive) and that the legislative measure is still relevant today.

The Directive is also seen as relevant for achieving policy goals beyond its key objectives, in particular the openness of government. Although sometimes confusion arises between traditional freedom of information legislation and the re-use of public sector data, stakeholders agree that the Directive is relevant and helpful in pushing for more transparent and open governments. The above findings are largely confirmed by data from the online public consultation. 87% of respondents agree that the PSI Directive is still relevant by ensuring the supply of PSI into the EU single market for PSI, with 82% convinced that it ensures sufficient usability (e.g. machine-readability) of data. Furthermore, 87% respondents agree that the Directive ensures fair market access (non-discrimination) for all re-users and 85% believe that it ensures transparency and accountability of public sector bodies. Thus, at this stage, the appraisal of the relevance of the PSI Directive seems very positive. At the same time, the interaction with stakeholders also suggests that there might be a number of challenges for the future, which could affect the relevance of the PSI Directive. These emerging and future stakeholders’ needs concern in particular: ·the scope of the Directive, which does not include a number of valuable datasets, such as for instance research data and data held by public undertakings in utility domains (energy, transport, waste etc.). Some Member States are legislating in these areas in view of making more data of public interest available, while others are not, which risks creating an uneven playing field across the EU. ·the lack of a clear direction concerning dynamic data and APIs. The 2013 rules concerning machine-readable and reusable format already facilitate the re-use of data, but these provisions are not sufficient anymore in a world in which data have become much more frequently re-used in real time. To extract value from such PSI it is imperative that public sector bodies make it available in automated ways, ideally via APIs. 2013 changes to the PSI Directive At the time of its adoption, the new PSI Directive provisions related to cultural data were relevant to the context and the needs of the main stakeholders. Four years later, the Directive remains relevant because, according to the stakeholders interviewed, the conditions that made the Directive relevant in 2013 have not radically changed in the meantime. In particular, cultural institutions still need an EU-wide framework for enabling re-use of their data (as this has not been fully put in place yet) and re-users’ demand for data and transparency is still stable if not increasing in certain domains. As for changes concerning charging, Member States confirmed that charging at zero or marginal costs is the new norm for the vast majority of public sector bodies. However, all Member States still use the exceptions to the marginal costs regime, even beyond the cultural sector that was brought into the Directive in 2013 under the more generous 2003 charging rules, which allow public sector bodies to charge above the marginal cost. Although it is generally acknowledged that lowering the costs for re-use of datasets can overall benefit the economy and society, public sector bodies charging beyond marginal costs of dissemination point out that they need the income to reinvest in the quality of their datasets: Open Data leads to higher tax income and to benefits throughout the public sector and the economy, but if the public sector bodies producing the data do not get the appropriate funding for maintaining the datasets, the overall quality and usability of the data will be at risk. The evidence suggests that since its adoption, the PSI Directive has retained its relevance to stakeholders’ needs. However, emerging trends such as the increase in dynamic data coming from sensors and IoT devices, and Member States actions in making data of public interest better available in new areas not covered by the Directive (e.g. public undertakings in transport and utility domains) might have an impact on its relevance for the future.



COHERENCE By and large, there is agreement among stakeholders that the Directive is largely coherent with the related legislation, although in the case of some instruments it would benefit from a degree of clarification. INSPIRE Directive The INSPIRE Directive establishes a legal and technical interoperability framework for the sharing (access and use) of spatial data held by public authorities for the purpose of environmental policies and policies and activities having an impact on the environment. Spatial data are amongst the most valuable datasets for both public re-use (to establish the environmental impact of policy measures, to adopt emergency plans in case of natural disaster) and for private re-use (for geo-localisation services for instance, for mobility and energy apps etc.) . This is one of the reasons why spatial information is covered by both the PSI Directive and the INSPIRE Directive. While the latter focuses technically on data access services, interoperability models and mandatory data-sharing between administrations, the former regulates the re-use of spatial datasets, including the conditions for re-use by third parties. Consulted stakeholders suggested that the interplay between the two Directives has been working relatively well in the past few years. They also mentioned during the interviews that the PSI Directive has had a positive influence on the lowering of charges for re-use of spatial data. From this perspective, the complementarity and coherence of the two Directives have played out well. However, for the future and due to the 2013 modifications of the PSI Directive, a few risks in terms of coherence could be identified, in particular as regards re-use conditions (e.g. charging and licencing). Some stakeholders argued that more guidance is needed to understand what is allowed for spatial data and services. It was noted that, in some cases, agencies managing spatial data are “borderline” in terms of compliance with the PSI Directive. Moreover, it was mentioned that issues related to interoperability of metadata could arise if the two communities (PSI and INSPIRE) do not coordinate sufficiently with each other: indeed, the INSPIRE metadata standard is a thematic extension of CKAN used by the PSI community. Besides these limited issues, which would benefit from some clarification, the coherence between the INSPIRE and the PSI Directives is confirmed in the results of the online consultation. Only 11% of respondents do not think that the PSI and INSPIRE Directives are well aligned. Almost a half of respondents (44%) believe that there are no problems in this area, while 45% do not have an opinion. EU data protection legislation

Information produced by the public sector often does not contain personal data (maps, meteorological information etc.). However, there are numerous cases in which PSI may contain personal data. Therefore the protection of personal data is an important aspect of handling PSI: data protection legislation shall be respected in such cases. This means that PSI law must be applied in full compliance with data protection law and that it does not create any exceptions to it – this is explicitly recognised by the current PSI Directive. Indeed, according to Article 1(4) “This Directive leaves intact and in no way affects the level of protection of individuals with regard to the processing of personal data under the provisions of Union and national law, and in particular does not alter the obligations and rights set out in Directive 95/46/EC”. This is reinforced in recital (11) of Directive 2013/37/EU, stating that it “should be implemented and applied in full compliance with the principles relating to the protection of personal data in accordance with Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data”. In 2013 the Article 29 Working Party has provided an opinion on the application of data protection law in the context of PSI. The objective of this opinion was to help ensure a common understanding of the applicable legal framework, and offer consistent guidance and best practice examples on how to implement the PSI Directive (as amended) with regard to the processing of personal data. The current Directive 95/46/EC will be repealed and replaced by the General Data Protection Regulation (GDPR) on 25 May 2018. This new data protection framework has been built on the existing legislation. The recital 154 of the GDPR states that “Directive 2003/98/EC of the European Parliament and of the Council leaves intact and in no way affects the level of protection of natural persons with regard to the processing of personal data under the provisions of Union and Member State law, and in particular does not alter the obligations and rights set out in this Regulation.” Moreover, Article 86 of the GDPR contains a specific provision on processing and public access to official documents. During the engagement with stakeholders some targeted challenges were raised by public sector bodies, for example about the use of appropriate techniques for pseudonomisation or anomysation in order to reconcile the provisions on the re-use of PSI with the right to the protection of personal data. This issue is, however, not specific for the PSI area. However, the stakeholders did not recognise during the public consultation any general issues related to the application of the PSI Directive and the legislation on the protection of personal data. Only 27% of respondents do not agree that the PSI Directive is well aligned with the current and future rules on the protection of personal data. The remainder are convinced that there are no issues in this area or do not have an opinion. The relationship between data protection law and PSI re-use – in the sense that both the public sector body and the re-user must comply with data protection law in full - is an established part of Union law. Database Directive The Database Directive adopted in 1996 provides for two types of database rights: copyright protection (Chapter II) and the sui generis database right (Chapter III). This new exclusive sui generis right is granted to database producers, for a period of 15 years, to protect their investment of time, money and effort, irrespective of whether the database is in itself innovative (“non-original” databases)”. Public sector bodies can hold such rights if they have invested “time, money and effort” in establishing a database and they want this investment to be protected. In practice, since the sui generis right, as all exclusive rights in the area of IPR, gives the right holder the capacity to authorise or refuse uses of its protected content, public bodies have sometimes argued that database protected content was not subject to the re-use obligations set out in the PSI Directive. In the absence of a clear provision regulating the relationship between the two instruments, this has created a situation where, in practice, public bodies may invoke (in some cases they have done so) the sui generis right to escape the application of the rules of the PSI Directive. This excludes the case where a database which is used by a public sector body in the course of its tasks was in fact created by a third party. There may seem to be a contradiction between the aims of the instruments: the PSI Directive aims at making as much public information available for re-use as possible (with the smallest number of exceptions possible), while the Database Directive “sought to create a legal framework that would establish the ground rules for the protection of a wide variety of databases in the information age” (also public datasets). Some stakeholders underlined how the different objectives do not seem to be fully compatible. The PSI Directive contains recitals that aim to provide clarity on the relationship between IPRs and the obligations under the PSI Directive. However, in practice legal certainty has not been achieved, at least as regards the interplay between the obligations under the PSI Directive and the sui generis database right held by public bodies. Case law suggests that such uncertain reading of recitals might be at play in practice. Also, in a French court case a judge decided that local authorities were allowed to deny the re-use of genealogy data to a website based on its sui generis right. This judgement was overturned by the Conseil d'Etat deciding that the sui generis right cannot prevent the re-use of data under the rules of the PSI directive. Also, in a 2012 judgment of the Court of Justice of the European Union, the Austrian state asserted sui generis right in the official companies register in an attempt to deny its re-use by private company. While this last case was not specifically about PSI but competition law , the case also highlights the uncertainty surrounding the applicability of the sui generis database right in cases of public data re-use. Moreover, various instances were reported by practitioners where access and re-use of public information such as population census data or official map and traffic data were prevented by authorities exercising their sui generis right. As a result, even if courts resolve the conflict of the directives to the advatange of the public information re-users, the transaction cost of litigation is a relevant factor to take into account. The misalignment is to some extent confirmed by the results of the online public consultation, in which 34% of respondents agreed that the PSI and Database Directives are well aligned, while 30% disagreed and 46% did not express any particular view. This trend is confirmed by the public consultation on the Database Directive where 29.8% of the respondent stakeholders agree that the Database Directive and the PSI Directive are not coherent legislation, while 31.5% agree that they are and 38.5% of the respondents did not express an opinion. At the same time and despite these occurrences, the intention has always been for the PSI Directive to take precedence over the sui generis right contained in the Database Directive. Therefore, it appears that clarification of the interplay between the two instruments, clarifying that the sui generis right should not stand in the way of the obligations of the PSI Directive, would be welcome. National access regimes The obligations of the PSI Directive build on the national access regimes, which can limit access to certain documents on the grounds on national security, commercial confidentiality etc. These access regimes are often not completely aligned. This has an impact on the availability of the information for re-use, since information that is not generally accessible in a Member State is not covered by the Directive. Nevertheless, the replies to the public consultation indicate that overall the PSI Directive is coherent with these regimes. Almost 72% of respondents agree with this statement or do not hold a specific view, while only 28% believe that there is lack of coherence between the PSI Directive and national access regimes that would be problematic.

Other legal instruments When discussing the PSI Directive and the main barriers for the re-use rule, stakeholders often mention the question of the barriers raised by the exception concerning third party copyright. However, in terms of legal clarity there is no particular inconsistency between PSI and copyright rules as the relation between the two regimes is transparent. Still, for the implementation of the Directive copyright raises a number of challenges, in particular due to different rules in Member States (for example in terms of the rights of individual public sector employees) and the burden of copyright clearance. Another issue raised in the consultation process concerns the Directive on the protection of undisclosed know-how and business information against their unlawful acquisition, use and disclosure (Trade Secrets Directive). If interpreted in an extensive manner, some PSI could fall in the category of trade secrets and would therefore benefit from an exemption to the PSI Directive. Indeed, Article 1 of the PSI Directive specifically excludes from its scope documents which are not subject to the national access regimes because they have “commercial confidentiality (e.g. business, professional or company secrets)”. The trade Secrets Directive was adopted in 2016 and has not been transposed yet in all Member States, but some stakeholders question the clarity of its relation with the PSI Directive and argue that guidance from the Commission in this respect is needed. 2013 changes to the PSI Directive As regards the extension of the scope to cultural data, there is a high level of coherence of the PSI Directive with the international framework of reference. Indeed, the PSI Directive provisions pursue objectives which are totally in line with those of UNESCO. In fact, they go even further in promoting accessibility and re-use of cultural information. Some challenges were identified with regard to the copyright framework and in particular the Directive on orphan works. Some stakeholders see the latter as too complicated and not necessarily consistent with the objectives of the PSI Directive in the area of cultural data. Indeed, the Directive on orphan works provides that “diligent search” is performed to clear the information/data on a specific orphan work for re-use. However, this diligent search is interpreted differently in different Member States and this might result in uneven opening up and re-using of similar cultural data. As regards the coherence of the 2013 changes concerning the charging provisions, the above mentioned issue on the relation with the INSPIRE Directive is relevant. Theoretically, these two regimes are perfectly complementary, as they have different purposes. Nevertheless, the question of the coherence between the INSPIRE Directive and the charging provisions of the PSI Directive has been raised by some stakeholders, so it might benefit from some clarification or formal alignment. The PSI Directive is overall coherent with other relevant EU legislation. Nevertheless a technical clarification confirming the relation with in particular the Database Directive and the INSPIRE Directive may be useful. In relation with data protection legislation the interaction is already very clear, in that the rules on re-use must be applied in full compliance with data protection legislation. EU ADDED VALUE The PSI Directive is based on Article 114 of the Treaty on the Functioning of the European Union (TFUE) and therefore strongly rooted in a Single Market logic. Based on the feedback received by stakeholders, the EU added value of the Directive is unquestioned. The stakeholders tend to agree that national initiatives in the field of PSI would have not been sufficient to address possible market distortions and market failures and to offer a level-playing field to all businesses. The instrument has helped to reduce the significant differences that existed between Member States. Replies to the online consultation support this claim. 87% of respondents agree that the PSI Directive has played a role in encouraging the national authorities to open up more public sector data, while almost 63% believe that it has facilitated access to PSI from countries other than the one where the person concerned lives. Similarly, almost 64% of the replies indicate that the PSI Directive has been conducive to the creation of an EU-wide market for products and services based on PSI, with less than 20% disagreeing. 2013 changes to the PSI Directive Stakeholders agreed on the high EU added value of the modifications brought in 2013 to the provisions on charging. National initiatives alone would not have been able to provide a level-playing field for all European re-users and public sector bodies with respect to the price of data. In the area of cultural data, stakeholders indicated that the Directive, together with practical measures such as the creation of Europeana, has laid a basis for the development of cross-border and EU wide services. However, it has also been argued that, as the Directive built on an already existing trend towards cultural heritage digitalisation and improvements in terms of data accessibility, its added value remained more limited than originally wished, especially considering that the biggest obstacles in terms of copyright regimes and financing of digitalisation are to be addressed at the national level mainly or in the framework of other EU initiatives (e.g. digitisation of cultural heritage or reform of the copyright ramework). Therefore, based on the data available at this stage, one could argue that the potential EU added value of the PSI Directive in terms of extension of the scope to cultural data has not been entirely translated on the ground due to a number of obstacles which could not be entirely and directly addressed by the Directive. Although some stakeholders expressed doubts regarding the capacity to streamline open data standards through a top-down approach, others underlined that the direction given by the EU in the Directive, and its concretisation in the 2014 guidelines on data formats, have brought some EU added value. Analysis and stakeholder views confirm that policy intervention at the Member States level could not have achieved the same results to the same extent as the PSI Directive. The overall EU added value of the instrument is unquestioned, although it is stronger for specific aspects of PSI re-use (charging) than for others (cultural data and formats). 6.Conclusions Evaluation of the PSI Directive, including the changes introduced in 2013, has shown that it continues to be an effective, efficient and relevant instrument that successfully contributes to the achievement of its main policy objectives, which are to stimulate the digital content market for PSI-based products and services, to stimulate cross-border exploitation of PSI and to prevent distortions of competition on the EU market. It has also been demonstrated that the PSI Directive does not impose a disproportionate burden on the stakeholders concerned. While it affects two groups of stakeholders: re-users (mostly from the private sector, including SMEs, micro enterprises and start-ups) and public sector bodies, it only imposes obligations (and possible resulting administrative burden) on the latter. It should also be borne in mind, however, that several obligations that the PSI Directive imposes on public sector bodies, notably those related to the practical arrangements of making data available, should be considered as part of an overall effort towards digitising administration rather than specific PSI Directive-related costs. Re-users, on the other hand, have benefited from the substantial simplification of the procedures for obtaining PSI access, which helps them save time and resources. However, the evaluation has also revealed that there are a number of areas that would need to be addressed in order to fully exploit the potential of PSI for the European economy and society.

First, recent technological progress has allowed public sector bodies to generate a wealth of dynamic datasets. Nevertheless, as shown in the evaluation, the provision of real-time access to such data via adequate technical means, such as APIs, and in conformity with recognised interoperable standards, is still an issue which limits the usefulness of such data for innovative and high-demand services. Second, while the Directive has definitely contributed to an increased supply of public sector data in the internal market, there are still vast amounts of high-value public data that remain shielded from its scope, such as research data or data from entities executing public sector tasks. Some Member States are moving faster and taking action to address this issue, but others are not. Third, the evaluation has shown that although the competition dimension of the PSI Directive works, there are some concerns as regards the continued existence of exclusive arrangements. These may take new forms, such as PSI data lock-in, leading to situations in which the re-use of data by one economic operator takes away the incentive for commercial re-use of the same dataset by other companies. Fourth, the Directive would benefit from some additional clarification as regards charging provisions. The evaluation has revealed that exceptions enabling public bodies to charge above the default upper limit of marginal cost of dissemination are less relevant than before. Finally, it has been observed that while the PSI Directive is coherent with the current legislation on related topics, its interaction with certain legal instruments could be clarified. This concerns the relation between the PSI Directive and the Database Directive and the relation between the PSI and INSPIRE Directives, in particular as regards charging mechanisms. Addressing the above challenges will ensure that the Directive will remain an effective, efficient and relevant policy measure with an even greater potential to generate EU added value for the European taxpayer.

ANNEX 1 – EVALUATION QUESTIONS Effectiveness ·Has the Directive ensured that competition in the internal market is not distorted? ·Has the Directive enabled the creation of jobs related to the digital economy and concerning digital contents? ·Have more data held by public sector bodies, including cultural heritage institutions, become available for re-use? ·Has PSI increasingly become a source of innovative services and products? ·Is PSI circulating freely across the EU and are cross-border applications based on such information easy to implement? ·Has PSI has become more affordable, including for start-ups and SMEs? ·Are exclusive agreements between public sector bodies and third parties used only exceptionally and are they strictly limited to the cases mentioned in the Directive (e.g. necessary for the provision of the public service)? Efficiency ·What ratio exists between the benefits achieved and the costs borne by the stakeholders concerned? ·Are the costs borne by the public sector bodies in implementing the Directive (e.g. adapting IT infrastructure, lower income from charges) offset by socioeconomic benefits of re-using data (e.g. creation of new digital applications and products, increased transparency)? ·Does compliance with the Directive require better data management processes of public institutions? Does this lead to cost savings and increased operational efficiency?

·In case a request for re-use is rejected and an applicant decides to appeal to the decision of public sector body, is the redress procedure swift, efficient and without excessive costs? Relevance ·To what extent is the overall Directive still relevant in 2017, 14 years after its first adoption and based on any possible evolution of stakeholders’ needs in the meantime? ·Is the PSI Directive still relevant by ensuring the supply of PSI into the EU single market for PSI? ·Is the PSI Directive still relevant by ensuring sufficient usability (e.g. machine-readability) of data? ·Is the PSI Directive still relevant by ensuring fair market access (non-discrimination) of all re-users? ·Is the PSI Directive still relevant by ensuring transparency and accountability of public sector bodies? Coherence ·Is the PSI Directive well aligned with and complementary to EU legislation on the protection of personal data? ·Is the PSI Directive well aligned with and complementary to the rules based on the INSPIRE Directive? ·Is the PSI Directive well aligned with and complementary to the rules based on the PAEI Directive? ·Is the PSI Directive well aligned with and complementary to the rules based on the Database Directive? ·Is the PSI Directive well aligned with and complementary to the rules based on national access regimes (rules which limit access to certain documents on the grounds of national security, commercial confidentiality, etc.)? ·Is the PSI Directive well aligned with and complementary to the EU copyright rules? EU added value ·Has the PSI Directive played a role in encouraging the national authorities to open up more public sector data? ·Has the PSI Directive facilitated access to PSI from countries other than the one where the person concerned lives? ·Is the PSI Directive conducive to the creation of an EU-wide market for products and services based on PSI?



ANNEX 2 – GLOSSARY

AI - Artificial Intelligence, intelligence displayed by machines, applied when a machine mimics cognitive functions that humans associate with other human minds, such as learning and problem solving. API (Application Programming Interface) - a set of technical protocols by means of which one piece of software asks another programme to perform a service. The service could be granting access to data or performing a specified function. Database Directive - Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, which harmonizes the treatment of databases under copyright law and introduces the sui generis right for the creators of databases which do not qualify for copyright. PSI data lock-in - arrangements where public sector bodies grant access to their datasets free of charge to one economic operator, for example in exchange for in-kind compensation, leading to a situation in which the re-use of such data by this economic operator would take away the incentive for commercial re-use of the same dataset by other companies. document - any content whatever its medium (written on paper or stored in electronic form or as a sound, visual or audio-visual recording), or any part of such content; synonymous with data. dynamic data - data from sensors (e.g. bus arrival times, meteorological data), whose economic value depends on its real-time availability. European Data Portal - a portal that harvests the metadata of PSI available on public data portals across European countries. Information regarding the provision of data and the benefits of re-using data is also included. formal open standard - a standard which has been laid down in written form, detailing specifications for the requirements on how to ensure software interoperability. high-value datasets - datasets which are particularly valuable assets for the economy and society at large. Access to and the re-use of such datasets can speed up the emergence of value-added information products and services, and also encourage participatory democracy. G8 members have identified 14 high-value areas – from education to transport, and from health to crime and justice – which should help unlock the economic potential of open data, support innovation and provide greater accountability. INSPIRE Directive - Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community , which aims to create a European Union spatial data infrastructure for the purposes of EU environmental policies and policies or activities which may have an impact on the environment. IoT - Internet of Things, network of physical devices, vehicles, home appliances and other items embedded with connectivity software which enables these objects to connect and exchange data.[ machine-readable format - a file format structured so that software applications can easily identify, recognize and extract specific data, including individual statements of fact, and their internal structure. marginal cost method - a principle applying to all charging for public sector data re-use in the EU, with some exceptions: public sector bodies may charge no more than the marginal cost of reproducing, providing and disseminating the documents. open data - data that is freely available to everyone to access and re-use as they wish, without restrictions from copyright, patents or other mechanisms of control. open format - a file format that is platform-independent and made available to the public without any restriction that impedes the re-use of documents. public undertaking - any undertaking over which the contracting authorities may exercise directly or indirectly a dominant influence by virtue of their ownership of it, their financial participation therein, or the rules which govern it. public sector body - a state, regional or local authority, a body governed by public law or an association formed by one or several such authorities or one or several such bodies governed by public law. re-use - the use by persons or legal entities of documents held by public sector bodies, for commercial or non-commercial purposes other than the initial purpose within the public task for which the documents were produced. sui generis database right - a sui generis database right is a property right, comparable to but distinct from copyright, that exists to recognise the investment that is made into compiling a database, even when this does not involve the creative aspect that is reflected by copyright. It is sometimes invoked by public sector bodies to justify their refusal to give access to their data.





ANNEX 3 – LIST OF LITERATURE USED (1)Merijn Backx, Gebouwgegevens redden levens. Toegankelijkheidseisen van gebouwgegevens in het kader van de openbare orde en veiligheid (scriptie Technische Universiteit Delft), 2003. BDO 199. (2)Bregt, A., L. Grus & D. Eertink (2014). Wat zijn de effecten van een open basisregistratie topografie na twee jaar?,Wageningen University: p. 49. (3)DECA, Danish Enterprise and Construction Authority, (2010). The value of Danish address data, http://www.adresseinfo.dk/Portals/2/Benefit/Value_Assessment_Danish_Address_Data_UK_2010-07-07b.pdf (4)DECA, Danish Enterprise and Construction Authority, (2017). The impact of open geographical data – follow-up study, http://sdfe.dk/media/2917052/20170317-the-impact-of-the-open-geographical-data-management-summary-version-13-pwc-qrvkvdr.pdf (5)Deloitte/Tech24. (2011, October). POPSIS Pricing Of Public Sector Information Study. Retrieved from https://ec.europa.eu/digital-single-market/en/news/pricing-public-sector-information-study-popsis-models-supply-and-charging-public-sector (6)European Commission. (2014). Study on eGovernment and the reduction of administrative burden, SMART 2012/0061, Final Report, https://ec.europa.eu/digital-single-market/news/final-report-study-egovernment-and-reduction-administrative-burden-smart-20120061 (7)European Commission. (2017, February 01). European Data Market. SMART 2013/0063. Final Report. Retrieved from https://a2528ba5-a-c3c32646-s-sites.googlegroups.com/a/open-evidence.com/download/repository/SMART20130063_Final%20Report_030417_2.pdf?attachauth=ANoY7cqfQr79x6OowAjJUkJgS-qZyc3dkemSFqn5LG_gZHt8nIJmQVPrRyUo5b4K17K5tCRBUFNSUo-xow8cw4_X5AhY_gOFF7LdqSlYvCG_HxdAVeHeEJYSA63vJLDnDTgdn7MZNkp8l6y6eGcUY-uNVcb1J9k1cFYoV-zbNauRXFw7fxoq8kKze10uSqUNhwUf6cpI6xxD6_xwdLqpvcMEvwhoCmDcL5F8NTz1UZmR2o4l2sV5kl6VfMzWyOMcvL90EtxlXCsG&attredirects=0 (8)European Commission: Study to support the review of Directive 2003/98/EC on the re-use of public sector information, SMART 2017/0061, Deloitte, January 2018 (9)European Data Portal: Open Data Maturity in Europe 2017 – Open Data for a European Data Economy, https://www.europeandataportal.eu/sites/default/files/edp_landscaping_insight_report_n3_2017.pdf

(10)European Data Portal: Analytical Report 9: The Economic Benefits of Open Data, December 2017. Retrieved from https://www.europeandataportal.eu/sites/default/files/analytical_report_n9_economic_benefits_of_open_data.pdf (11)Finnish Prime Minister's Office (2017), Avoimen datan yödyntäminen ja vaikuttavuus, https://www.etla.fi/wp-content/uploads/VNK_2017_40.pdf (12)Grus, L., A. Bregt & D. Eertink (2015). De effecten van open data BRT na 3 jaar, Wageningen University & het Kadaster: 7. (13)Houghton J., Costs and benefits of Data Provision, Report to the Australia National Data Service, Centre for Strategic Economic Studies, Victoria University, September 2011, http://www.ands.org.au/working-with-data/articulating-the-value-of-open-data/costs-and-benefits-of-data-provision (14)Harrison, T., & Sayogo, D “Transparency, participation and accountability practices in open government: A comparative study”. Government Information Quarterly, 2014, 31, pp. 513–525. (15)Katleen Janssen, Inspire and the PSI Directive: Public Tasks vs Commercial Activities?, Interdisciplinary Centre for Law and ICT (ICRI), K.U.Leuven, Belgium, 2005. https://www.law.kuleuven.be/citip/en/docs/publications/719inspire2f90.pdf (16)Katleen Janssen, “The influence of the PSI directive on open government data: An overview of recent developments”, Government Information Quarterly, Volume 28, Issue 4, October 2011, pp. 446-456. (17)Katleen Janssen & Ton Zijlstra, The new PSI Directive – as good as it seems?, 19 April 2013, , Open Knowledge International Blog, https://blog.okfn.org/2013/04/19/the-new-psi-directive-as-good-as-it-seems/ (18)Heli Koski, « Does Marginal Cost Pricing of Public Sector Information Spur Firms Growth ? », The Research Institute of the Finnish Economy, 2011, https://www.etla.fi/wp-content/uploads/2012/09/dp1260.pdf (19)National Observatory of Telecommunications and the Information Society (ONTSI), Public Corporate Entity Red.es, 2016, Characterization of the Spanish Infomediary Sector. Executive Summary, http://www.ontsi.red.es/ontsi/sites/ontsi/files/Characterization%20of%20the%20Spanish%20Infomediary%20Sector.%20Executive%20Summary.%20Ed.2016.pdf