awareness). In addition, public procurement data combined with performance indicators can nurture the spread of best practices in the public sector.<sup>25</sup> In spite of such promises a range of caveats apply. First, information available on official government and watchdog portals is typically on high level of aggregation (e.g. containing total contract value, but no information on the prices of individual inputs of deliverables) and often contain missing or erroneous information limiting its use for civil society monitoring. Especially as information on project execution progress and payments is typically missing from procurement portals, transparency may achieve little on the ground (Cingolani et al., 2015). Second, effective monitoring of complex transactions and projects, typical of public procurement, requires direct observability of performance and mismanagement. This may well be the case in construction, but in services such as education and training provision, lack of easy observability of mismanagement can render transparency ineffective (Banerjee et al., 2010). Third, costs of making data publicly available on transparency and watchdog portals may be substantial with oversight and monitoring costs even higher, significantly reducing net benefits or even turning them into negative.

# _b) Evidence_

Evidence on the effects of transparency and watchdog portals is very limited, it by and large incorporates non-systemic collections of suggestive facts and perception survey data (Šípoš et al., 2015). No experimental or similar study could be identified which would reliably establish causal relationships.

While no rigorous economic analysis underpins the above theoretically sound arguments, some data from the Slovakian watchdog portal provides suggestive evidence (Šípoš et al., 2015). The reach of public procurement information across the Slovakian population has increased dramatically from practically null to 11% of the total population claiming to have checked at least one contract online since 2011. In addition, media reporting of public procurement related scandals increased often making explicit use and reference to the watchdog portal: from 877 articles in 2003-2006 to 1765 articles in 2011-2014. Furthermore, where estimates exists, the additional cost of running transparency and watchdog portals is negligible not only for the managers of the portal but also for the organizations imputing data (Center for Global Development, 2014). These costs appear unimportant in particular when set against the billions of USD spent through them annually.

Given the high and quickly growing number of transparency and watchdog portals around the globe – Czech Republic, Hungary, Indonesia, Nigeria, Philippines, the Slovak Republic, Ukraine, and Vietnam, to name a few – rigorous evaluations are indispensable. In particular, understanding the conditions under which such portals can have a positive effect on access to public contracts is a key concern.

# 3. <u>Rule-bound or discretionary decision making</u>

# _a) Theory_

The dichotomy of rule-bound and discretionary decision making in public procurement is frequently reduced in the literature to the dichotomy of using (open) auctioning versus negotiation procedures in the bidding and award phases (Bajari et al., 2009).<sup>26</sup> However, it has broader ramifications than that implying impacts at the contract execution stage (Rasul & Rogger, 2015) as well as more generally referring to bureaucratic behavior. Hence, these set of tools are more broadly defined: one the one hand, discretionary or autonomous decision making in public procurement implies decision making freedom and flexibility of bureaucrats and procurement administrators throughout the procurement cycle bound only by general principles of good administration such as civil service code of conduct and organizational culture (Kelman,

> 25 One watchdog site with explicit purpose of spreading best practices in the Czech Republic: http://zindex.cz/

> 26 While this implies the general neglect of the issue of discretion in the contract execution phase which would place this discussion in section D; the general character of the issues rather leave it among interventions of more holistic character.

23