attempt at a meta-analysis was made, even though with the availability of more evidence we hope to be able to do so.

# **C. The context of public procurement reforms**

# 1. <u>The promise and challenge of Big Data in public procurement</u>

Big Data in public procurement, if appropriately harnessed, holds the promise of fundamentally transforming how procurement performance is understood and it can provide a vastly superior guide to effective policy decisions and implementation compared to our current knowledge. Such benefits are not automatic however, they also harbor considerable risks and require some investment.

Due to extensive regulations, the presence of multiple actors, and a demand for public security, public procurement has long been a data rich area of public spending. However, with the increasing use of electronic and online procurement tools, this rich set of administrative records have become more readily and more extensively available – giving rise to a data revolution experienced across many domains of social life (UN Global Pulse, 2012). This enables real-time data analysis using data sets tracking individual actions such as bids submitted to a tender, evaluation scores assigned, or invoices paid.

What fundamentally reconfigures our capacity to understand and govern public procurement systems is the move from individual records to a structured database.<sup>6</sup> This means that on top of the ability to identify and analyze a small set of tendering documents; governments, businesses, and citizens are becoming increasingly capable of systematically analyzing large swaths of procurement activities. In spite of the obvious informational basis and wide ranging benefits, making this move to a data rich approach has proven to be surprisingly challenging with some governments and international organizations even decreasing their ‘Big Data readiness’ rather than improving it (Fazekas & Saussier, 2018). On the one hand, building integrated data systems even if they only encompass the already collected information requires considerable investment. Linking disparate procurement data systems, standardizing data formats, and delivering reliable data management infrastructures are typically far from straightforward exercises with many frontline civil servants opposing change. Capturing past data, typically recorded in semi-structured text files, and building state-of-the-art data infrastructures going forward both require IT expertise and the understanding of complex data systems typically lacking in public sectors. On the other hand, systematizing data collection and publication had frequently revealed that reporting requirements are grossly neglected, making the most essential bits of data erroneous, missing or incomprehensible. The demand for using such data across the globe, as examples from Canada,<sup>7</sup> Czech Republic,<sup>8</sup> the European Union,<sup>9</sup> and Hungary reveal, makes the challenge of improving data quality even more pressing (Czibik et al., 2015). In addition, linking contract data to databases holding related information is also challenging but promises considerable benefits, most notably the ability to trace development outcomes back to procurement performance. Potentially linked databases include for example public financial management systems, treasury accounts of public bodies, company registry, financial and ownership data (including beneficial ownership registries gaining prominence recently), and information on sectoral outcomes such as student achievement or mortality (Fazekas & Tóth, 2016, 2014).

Big Data in public procurement also gives rise to the need for new, more advanced indicators which help diverse users in making sense of the often daunting variety of data; recall, public procurement includes purchases for anything ranging from nuclear power plants to school meals. Such new indicators of value for money and open access can potentially complement or in some cases replace traditional indicators of

> 6 - <u>http://www.open contracting.org/iodc15_open_contracting_reflections</u>

> 7 <u>https://sites.google.com/site/do101mtl/seao/iqd-1</u>

> 8 <u>http://www.profily.info/</u>

> 9 <u>http://www.open-contracting.org/digiwhist_big_data_meets_the_concerned_citizen</u> and <u>http://ec.europa.eu/internal_market/scoreboard/performance_per_policy_area/public_procurement/index_en.htm</u>

5