metadata: authors: 0: author: Fazekas,Mihály
1: author: Oliveira,Alexandre Borges De
2: author: Regös,Nóra
count: Latin America
docna: 0: docna: Lowering Prices of Pharmaceuticals, Medical
            Supplies, and Equipment : Insights from Big Data for Better
            Procurement Strategies in Latin America
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Pharmaceuticals & Pharmacoeconomics,Pharmaceuticals Industry,Public Finance Decentralization and
            Poverty Reduction,Legal Reform,Legislation,Regulatory Regimes,Judicial System Reform,Social Policy,Public Sector Economics,Legal Products,Public Health Promotion
teratopic: Social Development,Law and Development,Governance,Industry,Public Sector Development,Health, Nutrition and Population
abstracts: cdata!: Containing rapidly growing health care
            costs in the Latin American and the Caribbean region,
            especially amid the COVID-19 pandemic, requires an in-depth
            analysis of prices from a novel perspective. This paper
            documents hitherto understudied variations in prices paid
            for pharmaceuticals, equipment, and medical supplies within
            countries and markets. It also identifies effective
            procurement strategies for lowering prices within existing
            regulatory frameworks. The analysis uses public procurement
            data gathered by governments’ electronic procurement systems
            in nine countries and territories across the region. The
            data are uniquely detailed and complete, encompassing the
            minute detail of purchasing decisions and processes made
            across all regulated public entities in the study countries
            and territories. Traditional regression analysis and machine
            learning (random forests) methods are used to explain prices
            as a function of procurement decisions and outputs, such as
            the number of bidders. Based on in-depth discussions with
            policy makers, the paper also devises realistic policy
            interventions, which in turn can be used to estimate savings
            scenarios. First, the findings show that the prices paid
            vary greatly across and within countries. The latter is
            surprising given that the regulatory and institutional
            framework is largely fixed within each country. Second, a
            high proportion of within-country and -market variation can
            be explained by standard features of procurement policy
            implementation, such as the length of advertising tenders.
            Third, the explanatory models point to the potential for
            lowering prices across the region by about 14 percent by
            implementing low-level, yet impactful changes to how
            purchasing is done.
repnme: repnme: Lowering Prices of Pharmaceuticals, Medical
            Supplies, and Equipment : Insights from Big Data for Better
            Procurement Strategies in Latin America
display_title: Lowering Prices of Pharmaceuticals,
            Medical Supplies, and Equipment : Insights from Big Data for
            Better Procurement Strategies in Latin America
listing_relative_url: /research/2021/06/33142299/lowering-prices-pharmaceuticals-medical-supplies-equipment-insights-big-data-better-procurement-strategies-latin-america
topicv3: Public Health m1327519 635,e-Government Procurement m1327708 1880,Pharmaceuticals m1327740 1350,Health m1504661 3480,Coronavirus (COVID-19) m1939478 4828
publishtoextweb_dt: 2021-06-04T00:00:00Z
dois: 10.1596/1813-9450-9689
historic_topic: Social Development,Law and Development,Governance,Industry,Public Sector Development,Health, Nutrition and Population
origu: OPCS Procurement (OPSPR)
no_of_pages: 51
author_keys: 0: author_key: 458354
model_extractions: Policy Research Working Paper 9689 Lowering Prices of Pharmaceuticals, Medical Supplies, and Equipment Insights from Big Data for Better Procurement Strategies in Latin America _Mihály Fazekas Alexandre Borges de Oliveira Nóra Regös_ Latin America and the Caribbean Region June 2021
Policy Research Working Paper 9689

# **Abstract**

Containing rapidly growing health care costs in the Latin American and the Caribbean region, especially amid the COVID-19 pandemic, requires an in-depth analysis of prices from a novel perspective. This paper documents hitherto understudied variations in prices paid for pharmaceuticals, equipment, and medical supplies within countries and markets. It also identifies effective procurement strategies for lowering prices within existing regulatory frameworks. The analysis uses public procurement data gathered by governments’ electronic procurement systems in nine countries and territories across the region. The data are uniquely detailed and complete, encompassing the minute detail of purchasing decisions and processes made across all regulated public entities in the study countries and territories. Traditional regression analysis and machine learning (random forests) methods are used to explain prices as a function of procurement decisions and outputs, such as the number of bidders. Based on in-depth discussions with policy makers, the paper also devises realistic policy interventions, which in turn can be used to estimate savings scenarios. First, the findings show that the prices paid vary greatly across and within countries. The latter is surprising given that the regulatory and institutional framework is largely fixed within each country. Second, a high proportion of within-country and -market variation can be explained by standard features of procurement policy implementation, such as the length of advertising tenders. Third, the explanatory models point to the potential for lowering prices across the region by about 14 percent by implementing low-level, yet impactful changes to how purchasing is done.

This paper is a product of Latin America and the Caribbean Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at aoliveira@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Lowering Prices of Pharmaceuticals, Medical Supplies, and Equipment: Insights from Big Data for Better Procurement Strategies in Latin America

**Authors** : Mihály Fazekas (Central European University and Government Transparency Institute), Alexandre Borges de Oliveira (World Bank), Nóra Regös (Government Transparency Institute)

JEL classification: C53, D44, H51,H57, I11, C57, C55, C61 Keywords: auctions, Latin America, procurement, pharmaceuticals
# 1. Introduction

The COVID-19 pandemic has put a lot of pressure on already strained health care systems across the Latin America and the Caribbean (LAC) region (World Bank, 2020). Among others, the explosion of costs, such as prices paid for pharmaceuticals (Chernew & May, 2011), has contributed to failings in equitable and quality health care provision in the region. COVID-19 emergency states and the accompanying pressures to deliver critical goods and services fast have further contributed to price pressures in already fragmented markets.

Responding to these policy challenges, this study aims to

- document hitherto understudied variations in prices paid for pharmaceuticals, equipment and medical supplies within countries and markets; and

- identify effective procurement strategies for lowering prices within existing regulatory frameworks with the help of purchase-level explanatory models.

These ambitions are made possible by the unique data set compiled from public sources enabling price analysis at the very detailed, purchase level (i.e. within countries and markets over time). This represents a major departure from most prior studies which had to rely on market-level average prices across countries (e.g. Danzon & Chao, 2000). We make use of public procurement data as gathered by public authorities in 9 countries and territories across the LAC region through their electronic procurement systems. This data is uniquely detailed and complete, encompassing the minute detail of purchasing decisions and processes made across all regulated public entities in the study countries and territories.

Our sample of 7 countries (Ecuador, Brazil (federal), Paraguay, Panama, Uruguay, Peru, and Costa Rica) and 2 territories (Amazonas (Brazil) and Santa Catarina (Brazil)) is exceptional in its breadth; still, it misses out on important countries in the region and hence may represent a skewed sample. Nevertheless, the wide country coverage of our sample allows for making cross-country claims in the region, by exploiting the variance in terms of institutional and regulatory set-up. We make use of traditional regression analysis as well as machine learning (random forests) methods to explain prices paid for pharmaceuticals, equipment and medical supplies. Based on in-depth discussions with policy makers, we devise realistic policy interventions which in turn can be used to estimate savings scenarios.

First, we find that prices paid even for highly standardized pharmaceuticals vary greatly across as well as within countries. The latter is surprising given that regulatory and institutional frameworks are largely fixed within each country. Second, a high proportion of within-country and within-market variation can be explained by standard features of procurement policy implementation such as the length of advertising tenders. Third, our explanatory models point at the potential for lowering prices across the region by about 14% by implementing low-level, yet impactful changes to how purchasing is done.

2
# 2. Institutional set-up: Health sectors, procurement rules and purchasing discretion

Latin America and the Caribbean countries’ economies have been weakened by multiple adverse events throughout the past decades, such as extreme drop of commodity prices, rigorous financial constraints, and natural disasters (World Bank, 2020). These events also had a negative impact on social care systems, resulting in lower quality, unequal health care services. From 1990 onwards, countries in the region put great effort in changing the situation and introduced several health care reforms to reduce inefficiency, inequity and increase coverage of health systems, however there is still room for improvement (OECD, 2020). During the ‘Golden Decade’, between 2003 and 2013 (except the 2008 financial crisis), the region experienced stable economic growth, which led to higher public spending in the health care sector, although in the past 5-6 years a declining tendency is observable, contributing to political unrest in many countries in the region in 2019 (World Bank, 2020). In addition to the already nonoptimal conditions, COVID-19 further burdens these countries’ health care systems and their economies (World Bank, 2020). Weaker health systems and health services coverage are also indicated by the high level of out-of-pocket health expenditures, accounting for 34% on average of total health spending (OECD average is 21%) (OECD, 2020). Public health expenditure is low (3.8% of GDP) in the LAC region compared to OECD countries (6.6% of GDP). Public spending, the share of total health expenditure and compulsory insurance is 54.3%, while the OECD average is 73.6%, which deepens unequal access to health care services. Provision of higher quality health care services could also be achieved by more efficient spending (OECD, 2020). Low performance is also compounded by high levels of corruption, for example bribery rates in public health centers are around 11% in the region (OECD, 2020).

The market for pharmaceuticals is very heterogeneous with simple, low cost products, such as aspirins, and complex, high cost products, such as cancer therapeutics. Furthermore, some products are purchased in very large quantities while others in small quantities. It can be argued that there are many different sub-markets within the market for pharmaceuticals. In fact, the data set built for this work encompasses 235 products with price per unit varying from USD 9X10<sup>-7</sup> to USD 7,008. This diversity of products requires governments to develop an array of strategies and approaches to procurement if they are to maximize value for money and improve results in this complex market.

Our analysis revealed that governments have different strategies and approaches to procurement of pharmaceuticals, leading to varying outcomes. Procurement regulations in place in each jurisdiction as well as choices made by government officials in charge of implementing procurement activities are the two key factors driving results.

Procurement regulations and the institutional framework vary substantially across the jurisdictions included in this analysis, driving procurement activities and outcomes. Among these, the most notable are: 3
1. the existence of a centralized procurement agency or centralized approach to procurement of pharmaceuticals,

2. setup of umbrella or framework agreements, 3. requirements for use of competitive versus non-competitive procurement procedures - usually implemented through monetary thresholds.

The diversity of approaches within these three parameters were also significant: even when a centralized procurement agency or centralized procurement approach exists, some jurisdictions mandate their use to all health care delivery units (e.g. hospitals or local clinics) while other jurisdictions made these optional. Umbrella or framework agreements were used by some jurisdictions but not by others; and, among those that used umbrella contracts, there are significant differences in terms of mandatory use, number of supply choices in the umbrella contract, institutional arrangements to set up the umbrella agreement and so on. As for competitive, restricted or non-competitive procurement arrangements, the jurisdictions that are part of this work had material differences in terms of the monetary thresholds which are typically used to determine the use of competitive versus restricted or non-competitive procurement approaches. Furthermore, procurement regulations in all jurisdictions had numerous legal avenues allowing for non-competitive procurement even for larger purchases which would require competitive procurement based on monetary value. Finally, some jurisdictions favored the use of reverse auctions for competitive procurement while others relied solely on the traditional tender approach of requesting one sealed bid with the lowest price.

In addition to regulatory elements, government officials have considerable discretion to procurement strategies and make implementation decisions, even if they are constrained by national and local procurement regulations. Many of these choices reflect incentives as well as capacity of government agencies. Amongst the key elements for which government officials could make choices that impacted results were:

1. bundling of pharmaceuticals of similar nature as opposed to individual purchases for each item,

2. quantity requested in each purchase, even though this decision can be affected by budgetary allocations and planning procedures, 3. choice of procurement approach in jurisdictions where there is more than one method for competitive procurement (such as electronic versus face-to-face reverse auction, or reverse auction versus traditional bidding and many others), 4. time allowed for bidders to submit a bid, although the minimum time is always defined by regulations, 5. time allocated to evaluate and select the winner, 6. choice to purchase from a framework agreement or carry out a separate individual purchase in jurisdictions where such choice is allowed.

Also noteworthy are decisions and procedures outside the scope of procurement processes which had an impact on outcomes, such as: budget and planning procedures, payment terms and turnaround time to release it, court decisions mandating purchase of a certain brand of drug, clinical choices made for a specific brand or for purchase of generic drugs.

4
Variation in the organization of purchasing in terms of which agencies can buy independently, under which conditions, provided the variability of prices as well as procurement strategies to be exploited by the analysis. The key institutional differences are discussed below to offer insights into both the variations we see and the potential scope for policy change:

- Most jurisdictions had a piecemeal approach to procurement of pharmaceuticals with budget allocation specific to each government agency and autonomy to implement the budget. Consequently, procurement of pharmaceuticals was totally decentralized, irrespective of the type of product, resulting in multiple different tenders for the very same product from different government agencies. For agencies with larger budgets, it was very common to find multiple tenders within the same budget year for the very same product. This hands-off approach to procurement strategy resulted in substantial dispersion in prices for the very same product within a certain jurisdiction. This was the prevailing approach in most jurisdictions.

- One national and two subnational governments used procurement strategies to leverage the overall demand of the government to approach markets for supply of high value, complex pharmaceuticals as well as high volume, low complexity ones. These jurisdictions mandated the use of a framework contract to fulfill all demand for the products available within these contracts. A government agency would have to provide written justification if willing to purchase outside the umbrella contract, which was put out for bidding on a yearly basis. This approach resulted in smaller dispersion or, in the case of one of the subnational jurisdictions, no variation in prices paid for many of the products available in the yearly umbrella contract.

- One of the subnational jurisdictions had yearly framework contracts for many products as described above but allowed government agencies to purchase directly if the total award value was below a certain value threshold. Data showed varying uptake of the framework contract across government agencies and, in some cases, multiple small value purchases of the same product even by the same agency.

- One jurisdiction had a fragmented institutional setup for delivery of health services, in which government agencies were autonomous to purchase the products they need. The largest provider had one approach to procurement of pharmaceuticals while the other, much smaller providers followed a different strategy, in most cases purchasing without competition as their scale was small, and individual purchases rarely broke the threshold required for competitive procedures.

- A group of jurisdictions used reverse auctions, either electronically or face-to-face, to purchase the large majority of their demand for pharmaceuticals.

- One jurisdiction had procurement strategies affected by judicial orders that mandated procurement of a product from a specific manufacturer. This had a material impact on prices paid by the jurisdiction as the supplier benefited by the judicial order had almost full leverage over price. Most judicial orders required the government to fulfill the order in a matter of days, making it hard for the government even to try to negotiate down the initial price. These judicial orders required purchase of branded drugs in lieu of generic or

5
cheaper alternatives, and on many occasions came with clinical vouching from a health care provider.

# 3. Theoretical framework: Explaining unit prices at contract award

The determinants of pharmaceutical, medical supplies, and equipment prices have been studied in a wide variety of contexts and methodologies. For example, pharmaceutical prices have been found to be determined by patent expiration and generic producers’ market entry (Morton & Kyle, 2011), production costs (Berndt, 2002), a host of regulatory instruments such as international reference pricing (Wouters & Kanavos, 2017), or quality features (Danzon & Chao, 2000). Markets for medical devices or machinery have also been studied, albeit to a much lesser degree, for example by looking at market structure or bargaining power of market participants (Grennan, 2013). However, there have been relatively few theoretical or empirical investigations of purchasing decisions by public bodies vis a vis suppliers of pharmaceuticals, medical supplies and equipment. A recent notable such study looks at pricing transparency in the public procurement of selected drugs in 2 Brazilian states, Paraiba and Sao Paulo (Kohler et al, 2015).

Our theoretical framework takes note of this large body of evidence and looks at the least studied aspect of pricing pharmaceuticals, medical supplies and equipment: public purchases. Such an analytical focus also allows us to bring in and build on a hitherto disconnected literature on the performance of public procurement systems (de Oliveira et al, 2019; Kohler & Dimancesco, 2020). This literature has generated evidence on a diversity of determinants of prices at the point of contract award, some of which overlap with the public health literature briefly enumerated above: the price impact of bidder number, market structure, or centralization of purchasing (Petersen et al, 2020). While there is a host of price predictors largely neglected by the public health literature such as procedure type choice (e.g. open versus negotiated procedure), auction design choices (e.g. length of advertising the tender), or institutional quality (Baldi & Vannoni, 2017; Coviello & Mariniello, 2014).

Studying specifically the purchasing decisions made by a wide variety of public buyers across many countries, but within a relatively short time frame (the bulk of the data comes from 20142018), implies that many of the price determinants such as production costs or patent expiry can be assumed to remain largely unchanged from the perspective of our purchase level analysis. Holding such (for us) unobserved factors constant allows the analysis to concentrate on those observed factors which take center stage in public auctions and other types of tenders for standardized health care inputs. These observed price determinants such as market structure, auction design, or bidder features have been derived from at least one of the disciplines we build on: public health, health economics and public procurement. In addition, we also briefly draw on public administration research regarding the effect of administrative quality on prices.

6
The here outlined theoretical framework is derived from the goals of the study, hence it narrowly concentrates on the public procurement decisions and outcomes in health care, while being encompassing in that it covers a wide variety of procurement variables. Such narrow but comprehensive focus is suitable for our research goals because the underlying data is both of a very broad scope virtually including all health care-related regulated public tenders and of great depth capturing detailed information on tendering procedures, the actors and results.

The broadest outcome variable we aim to explain is value for money which is defined as the quality of goods or works obtained for a given procurement price. As reliable and comparable data on product quality is hard to obtain, our key focus falls on unit prices at the contract award for (largely) standardized goods. Hence, we define this main dependent variable of interest as _Unit price at contract award = total value of items contracted / standardized quantity of items contracted_ This definition of the dependent variable requires the units of measurement to be standard, at least within markets. This is because we can use market IDs as control variables in the regressions or features in the random forest models, hence making comparisons across purchases within markets rather than across markets. Naturally, units of measurement can vary from market to market (e.g. mg of paracetamol and liters for water). Nevertheless, price elasticities expressed in percent changes can still be compared across markets using our approach.

Using unit prices at contract award only approximates, and arguably imperfectly so, value for money. First, the focus on standardized goods and making comparisons with respect to market averages allows for keeping quality differences largely constant; even though some unobserved quality differences may remain. Second, prices at contract award do not take into account cost changes due to contract modification during the implementation phase. Third, payment and delivery schedules (e.g. delivering the same 1000 syringes is likely to have different unit prices if it is to be done by tomorrow versus in 5 months) may differ impacting both the value of provision and production costs injecting an additional unobserved heterogeneity to unit price comparisons.

In spite of such shortcomings, there is a wide ranging, policy-relevant, and methodologically diverse scholarship using unit prices in economics, political science, and public health (e.g. Bandiera, Prat, & Valletti, 2009; Collier, Kirchberger, & Söderbrom, 2015; Fazekas & Tóth, 2018; Lewis-Faupel, Neggers, Olken, & Pande, 2016; PricewaterhouseCoopers, 2013; Andrei Yakovlev, Vyglovsky, Demidova, & Bashlyk, 2015). While acknowledging shortcomings, we follow this literature and demonstrate unit prices’ applicability and usefulness for public procurement as well as health economics research.

On the explanatory factors’ side of the equation, the conceptual framework aims to be encompassing, capturing all major phases and actors of the purchasing process while also incorporating structural factors (Fazekas & Blum, 2021). Following de Oliveira et al (2019), we 7
group price predictors according to market features and main components of the tendering process such as specifications, actor characteristics (buyers as well as bidders), and outcomes (Table 1). Such simple grouping of factors helps us to develop a comprehensive measurement framework while clearly delineating alternative types of price determinants.

In addition, these features are amenable to policy intervention to different degrees facilitating evidence-based policy making in health care. By implication, explanatory factors are gathered into 2 main groups at the end of the analysis: those which are directly determined by policy makers such as purchasing officials (e.g. length of advertising a tender) and those which can only indirectly be influenced by policy (e.g. number of bidders). While it is relatively straightforward to classify each predictor according to this framework, in practice, some of them appear in combinations giving rise to relational features such as physical distance between the buyer and supplier (determining transportation costs among others).

8
Table 1. Summary of main groups of explanatory factors used in the analysis

|Group|Definition|Examples|Type|
|---|---|---|---|
|Market<br>characteristics|The technological and<br>competitive structure of<br>the market.|· Number of potential<br>suppliers<br>· Technological complexity|Structural|
|Tender<br>specifications|The conditions for<br>bidders to participate in<br>a tender.|· Length of advertising<br>bids<br>· Conditions of<br>participation|Directly policy<br>influenceable|
|Buyer<br>characteristics|Level of administrative<br>quality of the buying<br>office or agency|· Average time taken for<br>evaluating bids<br>· Buyer office location|Directly policy<br>influenceable|
|Bidder/supplier<br>characteristics|Company productivity<br>and capacity|· Company size<br>· Company headquarters<br>location|Indirectly policy<br>influenceable|
|Tender<br>outcomes|Intermediate and final<br>results of the tendering<br>process|· Number of bids<br>submitted<br>· Contract start date|Indirectly policy<br>influenceable|

_Source: de Oliveira et al, 2019_ Market or product group characteristics encompass those factors which describe the technological and competitive structure of the market in which the items are purchased. Most of these variables are in effect structural givens from the perspective of the analysis as they tend to be changing slowly or policy can only influence them at high cost. Procurement markets can be defined as a combination of product codes and geographical codes (Fazekas & Tóth, 2016) which capture fundamental differences, among others, by product complexity and technological characteristics, geographical characteristics (e.g. remoteness), and market concentration. For 9
standardized goods in health care but also beyond, unit prices are strongly influenced by such factors; for example higher population density increases road costs (100 people by square km leads to a 10% price increase) on a global roads construction sample (Collier et al., 2015), or higher social capital leads to lower goods prices in Italy (Bandiera et al., 2009).

Tender specifications capture all the conditions defining who can bid, under which procedural conditions and according to which assessment criteria and award decision rule. These variables are typically directly influenceable by policy makers without changes to the legal framework. Tender specifications variables are defined in the tender preparation and bidding stages of the procurement process. Among many, advertising tenders in a widely used, free, online portal tends to lower prices, such as in Italian public works tenders where the effect size is 7% higher winning rebates (i.e. discounts compared to the reference price) (Coviello & Mariniello, 2014); or longer term, fixed price contracts are more expensive in Russian sugar purchases while larger volumes lead to lower unit prices (Andrey Yakovlev, Bashina, & Demidova, 2014).

Buyer characteristics-related variables describe the level of administrative quality or capacity in the buying public entity such as a purchasing office or agency (Fazekas & Czibik, 2021). While many of these characteristics are directly influenceable by policy intervention such as capacity building trainings, many reforms may be challenging to implement such as giving more discretion to purchasing officials in the wake of public demands for accountability and impartiality. Among many others, procurement staff capacity lowers prices across many contexts (Best, Hjort, & Szakonyi, 2017), for example, in the US federal bureaucracy, one standard deviation increase in competence decreases cost overruns by 29% (Decarolis, Giuffrida, Iossa, Mollisi, & Spagnolo, 2020); or increasing procurement officers’ autonomy compared to their auditors reduces prices of standard goods by 9% in Punjab, Pakistan (Bandiera, Best, Khan, & Prat, 2019).

Bidder and supplier characteristics capture the key determinants of companies participating in public procurement which determine their ability to offer low prices such as productivity and capacity to deliver to requirements on time. As suppliers are selected through tenders and results at least partially depend on the choices companies make, these characteristics tend to be only indirectly influenceable for policy makers. Among many factors, company location and size are likely to influence prices while company risks such as tax haven registration or political connections also typically impact prices (Fazekas, Tóth, & King, 2016). For example, in India, roads built by politically connected contractors are on average 11% more expensive (Lehne, Shapiro, & Eynde, 2018).

Tender outcomes correspond to the intermediate or final results of the tendering process which naturally influence unit prices at contract award. These factors tend to be only indirectly influenceable by policy makers given the important role played by bidder decisions and interactions. A widely cited factor, the number of bidders, plays a highly influential role in determining prices under non-collusive conditions (Fazekas & Kocsis, 2017; Andrey Yakovlev et al., 2014).

10
# 4. Methodology

## 4.1 Data

Public procurement data underlying the analysis must be high quality, wide scope and high resolution, including information on individual items purchased and their unit prices. We obtained the data sets directly from the official data holders such as national public procurement authorities. Data sets contained structured information on tendering processes such as procedure type used, tendering outcomes such as number of received bids, and information of actors such as buyer and bidder names. More on indicators in section 4.2. Given our goal of looking at health care markets across countries, the biggest challenge of database building was to harmonize data sets coming from different legal and IT systems, for example by matching product classifications from different countries.

When compiling the data set for the analysis, we started from the full public procurement data sets in each country and territory: we compiled a data set of 7 countries and 2 territories (2 states of Brazil). As a first step of database creation, we selected all tenders which relate to the health care sector in the source data sets (Table 2). Second, we narrowed down the list of selected health care-related tenders to those which belong to an overlapping product category across all 9 data sets. This harmonization process resulted in a considerable reduction of our database size from about 128,000 tenders to 36,611 tenders, basically due to incommensurate national product classifications at the very detailed level we needed for the unit price analysis. The data was available to the research team largely for the period between 2012 and 2018, with considerable differences across countries. While each data set was defined on the level of item or purchase, in one country data structure differed. Here, the unit of observation was purchase orders within framework agreements which had to be aggregated to unique items within tenders to make them comparable to the other data sets.

11
_Table 2. Overview of data used by country and territory_

|Country|Total number of health care-<br>related tenders in country<br>data set|Total number of health care-<br>related tenders in combined<br>health care data set|Years covered|
|---|---|---|---|
|Ecuador|22609|22609|2013-2017|
|Brazil<br>(federal)|14108|2140|2014-2016|
|Amazonas<br>(Brazil)|9030|2797|2014-2018|
|Santa<br>Catarina<br>(Brazil)|1348|948|2013-2018|
|Paraguay|2899|830|2012-2016|
|Panama|56738|5439|2014-2018|
|Uruguay|12319|1008|2014-2018|
|Peru|9217|686|2015|
|Costa Rica|517|154|2016-2017|
|Total|128785|36611||

Given that the key precondition for unit price analysis is the use of sufficiently detailed and standardized product categories which are in addition comparable across 7 countries, we offer further details on the product codes and their processing. We identified health care-related tenders included in the initial, broad data sets based on product classification codes and higher level descriptions as defined in Annex A, Table A1. The health care-related product code harmonization was based on a host of classification schemes as used in the national data sets following national legislation (Table 3). In the absence of a generic, reference categorization, we considered the Ecuadorian nomenclature as the master reference as it was the most comprehensive for the health care products analyzed in the study. We identified the equivalent classification levels in all data sets by using a semi-automated matching method with extensive manual crosschecks. We excluded unmatched categories from the current analysis to enable cross-country pooled analysis. While a large number of categories and tenders were excluded as 12
a result of this harmonization process, it is not considered problematic as the majority of them referred to chemicals and laboratory equipment which are out of focus of the study<sup>1</sup> .

> 1 for a detailed overview see:https://docs.google.com/spreadsheets/d/1IBxWlGteyrJHOzxovSpWOrIg9K6337ZHchIygNYMRg/edit#gid=1534722942

13
_Table 3. Product harmonization across countries, health care-related items_

|**Country**|**Classification**<br>**scheme**|**Level of**<br>**observation**|**Total number of**<br>**categories in country**<br>**data set (at level of**<br>**obs)**|**Total number of**<br>**categories**<br>**overlapping with**<br>**master file (Ecuador)**|
|---|---|---|---|---|
|Ecuador|CPC - Clasificación<br>Central de<br>Productos|<br>Level 5 - CPC Level<br>5|222|222|
|Paraguay|UNSPSC - United<br>Nations Standard<br>Products and<br>Services Code|Level 4 - Descripción<br>ítem nivel4<br>(Commodity)|5122|217|
|Panama|UNSPSC - United<br>Nations Standard<br>Products and<br>Services Code|Level 4 - Nombre<br>Rubro (Commodity)|5122|217|
|Uruguay|SICE - Sistema de<br>Información de<br>Compras y<br>Contrataciones del<br>Estado|Level 5 - Descripción<br>Artículo|1097|166|
|Peru|CPC - Clasificación<br>Central de<br>Productos|Level 4 - Commodity|1763|203|
|Brazil<br>(federal)|Catálogo de<br>Materiais e Serviços|<br>Level 3 - Padrao|2445|181|
|Amazonas<br>(Brazil)|Catálogo de<br>Materiais e Serviços|<br>Level 3 -<br>Classification<br>Commodity|2211|195|
|Santa<br>Catarina<br>(Brazil)|Catálogo de<br>Materiais e Serviços|<br>Level 3 - Item<br>Classification<br>Description|5996|205|
|Costa Rica|UNSPSC - United<br>Nations Standard<br>Products and<br>Services Code|Level 5 (free text) -<br>DescProducto|692|92|

14
In addition to product classification harmonization, we harmonized unit prices, contract values and procurement methods. Price variables without VAT were converted into international constant USD using official World Bank PPP exchange rates.<sup>2</sup> A host of national procurement methods were grouped into open, restricted, non-competitive and other types. We marked framework agreements separately.

# 4.2 Indicators

When operationalizing the groups of explanatory factors and individual indicators of the theoretical framework we aimed to be as encompassing as possible while considering the limitations of the data set. Hence, we define a large array of indicators and let ‘the data decide’ on which ones are powerful predictors of unit prices (Table 4). Nevertheless, each of these indicators are policy relevant and derive from the literature which is based on tested economic and public administration theory using qualitative as well as quantitative methods. The interpretation based on the relevant literature is discussed in the results section.

> 2 https://data.worldbank.org/indicator/PA.NUS.PPP

15
Table 4. Summary of variables used in the analysis, LAC region

|**Type**|**Group**|**Variable name**|**Types**|**mean/ most**<br>**frequent**<br>**values***|**std.div**|**N(non-**<br>**missing)**|
|---|---|---|---|---|---|---|
|**DV**|**-**|Unit price|continuous|8512.9|453443.4|284,872|
|**Structur**<br>**al**|**Market**<br>**charact**<br>**eristics**|Market ID: reflecting<br>product code (1...235)|categorical|||287,041|
|||Year of contract award<br>(2012- 2018)|categorical|2014||287,041|
||**Buyer**<br>**charact**<br>**eristics**|Buyer type (independent<br>agency, ministry, etc)|categorical|National<br>gov.||286,634|
|||Buyer location (region)|categorical|||287,041|
|**Directly**<br>**policy**<br>**influenc**|**Tender**<br>**specific**<br>**ations**|Month of spending<br>(January, February, etc.)|categorical|July||33,155|
|**eable**||Procedure type (1 - fully<br>competitive; 2-restricted,<br>etc.)|categorical|Fully<br>competitive||279,720|
|||Advertisement period<br>length (days)|continuous|15.88|17.66|34,923|
|||Decision period length<br>(days)|continuous|29.80|33.07|30,491|
|||Failed tenders (%)|continuous|16.01|19.42|22,398|
|||Framework agreement<br>(Y/N)|binary|Yes||287,041|

16
||Product bundling|continuous|34.83|38.54|287,041|
|---|---|---|---|---|---|
||Quantity of purchased<br>goods (number of units)|continuous|23198.31|105542.8|287,041|
|**Indirectl**<br>**y policy**<br>**ifl**<br>**Bidder/**<br>**supplier**<br>**ht**|Buyer-supplier from the<br>same state (Y/N)|binary|No||282,028|
|**nuenc**<br>**eable**<br>**carac**<br>**eristics**|Supplier size (micro, small,<br>large company)|categorical|large||258,341|
||Supplier specialisation:<br>number of markets the<br>company supplies|continuous|83.87|39.65|284,718|
|**Bidding**<br>**outcom**|Number of bidders|continuous|4.32|7.21|48,703|
|**es**|Annual winner market<br>share (%)|continuous|57.04|38.47|284,614|
||Annual winner share in<br>buyer spending (%)|continuous|7.01|14.74|284,614|

_Notes: * most frequent values for categorical variables._

# 4.3 Methods

Given the unique scope of the data set we compiled, i.e. item-level health care purchases for 7 countries over many years, we set out to conduct both descriptive and explanatory analyses. The goal of the descriptive analysis is simply to demonstrate the variation in unit prices across as well as within countries. To this end we offer distribution graphs as well as variance decomposition by territory. Given the focus on average prices of most of the previous research, demonstrating the large within country and market variation in prices is both interesting on its own and serves as a motivation for our further analysis.

In order to identify effective procurement strategies improving value for money, we also conduct explanatory analyses which incorporate the wide array of indicators listed above. We compare traditional regression methods (Ordinary Least Squares) with random forests in order to identify the model with the best fit. Once the best model is identified, we can derive feasible changes in the predictor parameters, i.e. policy scenarios, which decrease unit prices in the models.

17
Given that the goal of the analysis is to explain variation of prices within each product market across the region, only those markets are considered which have adequate levels of within-market price variability. In our pooled analysis of the whole LAC region, we only retained those markets with at least 100 items awarded worth at least 10,000,000 USD in 2012-2018.

The first modeling approach to unit prices brings together all major explanatory factors into a single ordinary least squares (OLS) regression model. Such a comprehensive model allows for system-wide price predictions and simulating hypothetical scenarios. The following linear regression model for log unit prices of standardized products was estimated at the level of item purchased:

# Log(Pri) = αi +  ꞵ 1*X1i + ꞵ 2 * X2i + ꞵ 3 * X3i + εi

Where _Log(Pri)_ represents the natural logarithm of unit price for the _i_ th item purchased; _X1i_ stands for the set of directly policy influenceable predictors for the _i_ th item purchased such as the choice of procurement method; while _X2i_ represents the set of indirectly policy influenceable predictors for the _i_ th item purchased such as the number of bidders. _X3i_ denotes the set of control variables accounting for structural factors not amenable to policy intervention for the _i_ th item purchased such as the year of purchase. εi stands for the error term of the regression model.

For the sake of simplicity, we opted for a straightforward ordinary least squares estimation of the regression parameters which is a computationally efficient estimator able to handle models run on millions of records while containing hundreds of explanatory variables. Including a battery of fixed effects for structural factors such as market and year allows for building models with exceptionally high explanatory power: across the countries and territories analyzed, R2 ranges between 0.7 and 0.8. In spite of the linear functional form, the modeling framework is able to account for a range of nonlinear relationships by adopting a logged unit price dependent variable and also looking at alternative formulations of the independent variables, for example taking deciles of market concentration as a set of dummies rather than as a continuous scale.

Such a simple modeling framework cannot fully account for a range of complex relationships between factors on multiple levels such as market and tender level factors or explanatory factors influencing not only the dependent variable but also each other (e.g. advertising tenders not only influences prices directly but also through the number of bidders). While a full Multilevel Modeling strategy using country, product market and year levels is unfeasible on computational grounds, we systematically explored key interactions based on prior literature (de Oliveira et al, 2019). Hence, we will also report OLS results with such interactions to gauge the improvement in predictive power they confer.

The second main models built for explaining unit prices were Random Forests (RF) (James et al, 2015). RF estimation is based on intuitive tree-based models which split the sample in sequences minimizing prediction error. RF is an ensemble method which means that the eventual prediction of an RF model is based on aggregating over a large number of decision trees, each of which is 18
constructed using randomly varying parameters (randomly drawn samples of observations and predictors). Our RF models make use of the same wide array of explanatory variables as OLS models.

Given the large number of models we develop, we have to decide on how to choose between them. We consider hard and soft criteria for model choice. The hard criteria for model selection is explanatory power, in particular the mean squared error of the model on an ‘unseen’ test data set (25% random sample of the data). That is testing the prediction accuracy of models on a data set we did not use for developing the models which helps avoiding model overfitting. The soft criteria for model choice, carrying considerably lower weight, is its interpretability and the coefficients’ and partial impacts’ fit with prior literature.

The 2 sets of explanatory models, OLS and RF, both carry the advantage of exceptionally wide scope and computational efficiency, even though we cannot claim that the estimated conditional probabilities are causal. Nevertheless, we rely on strong, well-tested theories underpinning the choice of explanatory factors as well as their expected price impacts which together tentatively point at a causal story. In addition, the exceptionally high explanatory power of the models (R2 around 0.7-0.8), unusual for micro-level quantitative models, lends support to the claim that omitted variable bias does not plague our estimations.

Finally, once the best model is built, we re-estimate it on the full sample and explore the relationship between individual predictors and unit prices in detail. Upon this exploration and interpretation we devise a set of policy relevant and feasible policy scenarios, that is modifications to the input parameters, which allow for estimating the likely savings to be made from better procurement strategies in health care purchasing across LAC, pre COVID-19 pandemic.

19
<!-- Start of picture text --> 0<br>°<br>oO -<br>10<br>~ Oo<br>lo -<br>®<br>oO<br>Oo<br>oO<br>oO — = - bn a<br>ie<br>°<br>oO<br>10 -5 0 5 10 -10 -5 0 5 10 -10 -5 0 5 10<br>Deviation from market average(log scale)<br><!-- End of picture text -->
<!-- Start of picture text --> ae Trr ast uruin,FL] Aumoaicalical li n<br>BL<br>56 ite)<br>55000 oy saa 3 5t ae<br>saute jeg $4 515 53705 4008 5.6<br>S292<br>54 ite) 5a<br>ea Sipe<br>><br>41000 r<br>io La —$0] ie] —Sn ——S001<br>ss Clarithranyycin Ibuprofen<br>5255 $20<br>Pet 5156y LE 4 Ld 52.00<br>$a; Pa Py - $19 t 4 aL<br>S400 am $1.00<br>rt]<br>Te :<br>50.0 04 Baer)<br>$a 5.0.0}<br>Sodium chloride<br>‘+ 2<br>an |<br>ee] 2p<br>ies in<br>Si gif<br>eooe —— al —io ie- a hs pote$4<br><!-- End of picture text -->
The controls only model accounts for 77% of total price variation. Interestingly, there is some decrease in average prices between 2012 and 2018 (Table 1, Model 1). Adding directly policy influenceable features lifts explanatory power to 87%, i.e. these factors account for an additional 10% point of price variance (Table 1, Model 2). This model identifies relationships which confirm prior research (De Oliveira et al, 2019). For example, purchasing in larger quantities considerably decreases prices: while the average price is 13 USD in the 5th decile of purchased quantity (on average 862 units bought together), prices drop to 9 USD in the 6th decile (on average 1681 units bought together). Similarly, prices considerably drop in framework agreements: the average prices in framework agreements are predicted to be 21 USD, while 52 outside of it. Adding indirectly policy influenceable factors increases explanatory power to 89%, a small but relevant improvement (Table 1, Model 3). We find a further set of relationships, aligning with prior research. For example, decreasing market concentration is associated with lower prices: when the supplier’s market share drops from the highest decile (76% market share on average) to the 5th decile (27% market share on average), average prices are predicted to decrease from 89 USD to 59 USD. Similarly, local firms (registered in the same state as the buyer) tend to offer somewhat lower prices, on average 36 USD, compared to non-local firms offering on average 43 USD.

22
Table 5. Simple OLS regression model, training data set, LAC region

|Model<br>**1.Controls**|**2.Controls+Direct**<br>**policy**|**3.Controls + Direct +**<br>**Indirect policy**|
|---|---|---|
|Predictors/dependent variable|Logitem unitprice(|USD)|
|**Avg. Failed tenders, **Baseline: 0-67%|||
|Avg. Failed tenders=67.5-100%|-0.0620***|-0.412***|
||(0.000)|(0.000)|
|Avg. Failed tenders=missing|-2.539***<br>(0.000)|-2.904***<br>(0.000)|
|**Avg. Decisionperiod, **Baseline: 0-8 days|||
|Avg. Decision period=9-10 days|0.0812***|0.134***|
||(0.000)|(0.000)|
|Avg. Decision period=11-17 days|0.297***|0.396***|
||(0.000)|(0.000)|
|Avg. Decision period=18-39 days|0.500***|0.560***|
||(0.000)|(0.000)|
|Avg. Decisionperiod=40-358 days|0.492***|0.600***|
||(0.000)|(0.000)|
|Avg. Decisionperiod=missing|-0.397***|-0.518***|
||(0.000)|(0.000)|
|**Advertisementperiod length, Baseline:**1-7 days|||
|Advertisementperiod=8-12 days|0.0649***|0.0181|
||(0.000)|(0.135)|
|Advertisementperiod=13-183 days|-0.0957***|-0.0872***|
||(0.000)|(0.000)|
|Advertisementperiod=missing|-0.853***|-0.737***|
||(0.000)|(0.000)|
|**Procedure type, Baseline:**Fullycompetitive|||
|Restricted competition|-0.787***|-0.568***|
||(0.000)|(0.000)|
|Non-competitive|-0.546***|-0.429***|
||(0.000)|(0.000)|
|Other|-0.281***|-0.506***|
||(0.000)|(0.000)|
|Missing|0.777***|0.626***|
||(0.000)|(0.000)|
|**Framework agreement, Baseline: No**|||
|Framework agreement=Yes|-0.937***<br>|-0.993***<br>|
||(0.000)|(0.000)|
|**Month of spending, Baseline: December**|||
|January|-0.0614*|0.110***|
||(0.032)|(0.000)|
|February|0.0738**|0.126***|
||(0.002)|(0.000)|
|March|-0.150***|-0.0723***|
||(0.000)|(0.000)|
|April|-0.0287|0.0581**|
||(0.183)|(0.004)|
|May|-0.258***|-0.115***|
||(0.000)|(0.000)|
|June|-0.247***|-0.0637***|
||(0.000)|(0.000)|
|July|-0.204***|-0.0992***|
||(0.000)|(0.000)|
|August|-0.0670***|0.0463**|
||(0.001)|(0.009)|
|September|-0.00289<br>|0.0486**<br>|
||(0.880)|(0.006)|
|October|-0.0197|0.0155|

23
|November|(0.291)<br>-0.135***|(0.371)<br>-0.0719***|
|---|---|---|
||(0.000)|(0.000)|
|Missing|3.001***|3.888***|
||(0.000)|(0.000)|
|**Product bundling, Baseline: 1-3 items**|||
|4-8 items|-0.550***|-0.512***|
||(0.000)|(0.000)|
|9-14 items|-0.460***|-0.374***|
||(0.000)|(0.000)|
|15-22 items|-0.575***<br>|-0.479***<br>|
||(0.000)|(0.000)|
|23-27 items|-0.400***<br>(0.000)|-0.297***<br>(0.000)|
|28-35 items|-0.361***<br>(0.000)|-0.307***<br>(0.000)|
|36-38 items|-0.400***|-0.382***|
||(0.000)|(0.000)|
|39-50 items|-0.599***|-0.420***|
||(0.000)|(0.000)|
|51-111 items|-0.386***|-0.431***|
||(0.000)|(0.000)|
|112-154 items|-0.331***|-0.343***|
|**Quantity ofpurchasedgoods, Baseline: 1-24**|(0.000)|(0.000)|
|Purchasedquantity=25-200|-2.629***|-2.661***|
||(0.000)|(0.000)|
|Purchasedquantity=201-582|-3.497***|-3.491***|
||(0.000)|(0.000)|
|Purchasedquantity=583-3000|-4.130***|-4.073***|
||(0.000)|(0.000)|
|Purchased quantity=3001-10000|-4.363***|-4.397***|
||(0.000)|(0.000)|
|Purchased quantity=10001-39110|-4.696***|-4.719***|
||(0.000)|(0.000)|
|Purchasedquantity=39111-100000|-5.175***|-5.156***|
||(0.000)|(0.000)|
|Purchasedquantity=100001-319146|-5.515***|-5.543***|
||(0.000)|(0.000)|
|Purchasedquantity=319147-1000000|-5.996***|-6.065***|
||(0.000)|(0.000)|
|Purchasedquantity>1000000|-6.760***|-6.904***|
||(0.000)|(0.000)|
|**Supplier specialisation, Baseline:**1-8 markets|||
|Supplier is on 9-14 markets||-0.0291**|
|||(0.009)|
|Supplier is on 15-52 markets||-0.367***|
|||(0.000)|
|Supplier is on 53-69 markets||-0.671***|
|||(0.000)|
|Supplier is on 70-86 markets||-0.904***|
|||(0.000)|
|Supplier is on 87-98 markets||-0.750***<br>|
|Supplier is on 99-109 markets||(0.000)<br>-1.005***|
|||(0.000)|
|l k||***|
|Suppier is on 110-116 marets||-1.114<br>(0.000)|
|Supplier is on 117-123 markets||-1.317***|
|||(0.000)|
|Supplier is on 124-139 markets||-1.243***|
|||(0.000)|

24
|Missing<br>1.414***<br>|
|---|
|(0.000)<br>**Buyer-supplier location, Baseline: No**|
|Buyer-supplier location: Yes<br>-0.200***|
|(0.000)|
|Buyer-supplier location: Missing<br>0.554***|
|(0.000)|
|**Company size, Baseline: large**|
|medium<br>0.0786***<br>|
|(0.000)|
|small<br>0.0719***<br>|
|(0.000)<br>Missin<br>0973***|
|g<br>.<br>(0.000)|
|**Market concentration(by country), Baseline:**0-6.1%|
|Market concentration=0.04-18%<br>0.722***|
|(0.000)|
|Market concentration=0.12-59.4%<br>1.014***|
|(0.000)|
|Market concentration=0.26-59.4%<br>1.067***|
|(0.000)|
|Market concentration=0.65-84.6%<br>1.128***|
|(0.000)|
|Market concentration=1.3-98%<br>1.091***|
|(0.000)|
|Market concentration=2.9-100%<br>1.113***|
|(0.000)|
|Market concentration=5.9-100%<br>0.984***|
|(0.000)|
|Market concentration=12.7-100%<br>1.387***|
|(0.000)|
|Market concentration=28.7-100%<br>1.516***|
|(0.000)|
|**Buyer spending concentration (by country), Baseline:**0-1.2%|
|Buyer spending conc.=0.01-2.5%<br>0.416***|
|(0.000)|
|Buyer spendingconc.=0.06-4.3%<br>0.574***<br>(0000)|
|.<br>Buyer spendingconc.=0.2-7.7%<br>0.747***|
|(0.000)|
|Buyer spendingconc.=0.5-12.7%<br>0.855***<br>|
|(0.000)|
|Buyer spendingconc.=0.9-24.6%<br>1.023***|
|(0.000)|
|Buyer spendingconc.=1.7-53.5%<br>1.200***|
|(0.000)|
|Buyer spendingconc.=2.5-89%<br>1.352***|
|(0.000)|
|Buyer spendingconc.=4.9-100%<br>1.637***|
|(0.000)|
|Buyer spendingconc.=10-100%<br>1.723***<br>|
|(0.000)<br>**Bidder number, Baseline: 1-2**|
|Bidder number=3-7<br>-0.197***|
|(0000)|
|.<br>Bidder number=8-75<br>0.110***<br>(0000)|
|.<br>Bidder number=missing<br>-0.389***|
|(0.000)|
|Constant<br>Y<br>Y<br>Y|
|**Buyer type**<br>Y<br>Y<br>Y<br>**Buyer region**<br>Y<br>Y<br>Y|

25
|**Contract awardyear**<br>Y|Y|Y|
|---|---|---|
|**Market ID**<br>Y|Y|Y|
|Observations<br>284448|284448|282853|
|R-squared<br>0.769|0.868|0.886|
|p-values inparentheses *p<0.05 **p<0.01 ***p<0.001|||

Building on the simple, still comprehensive OLS model, it is possible to refine the analysis by allowing greater room for cross-country differences. The OLS models in Table 5 enforced the same slope or response intensities on all countries even though some variation by country in predictor definitions was allowed (e.g. defining deciles country by country rather than uniformly across all countries). This may not fit the structure of the data in all cases which may cause some relationships to be much weaker than prior research looking at individual countries. Hence, in cases where the initial findings warrant, we will allow for slopes and intercepts to vary country by country (For full details see Annex B).<sup>3</sup> For example, the number of bidders showed an ambiguous impact on prices in the simple OLS models of Table 5. However, when we allow for bidder numbers to have a different impact function across countries, different non-linear effects arise (Figure 3). In some countries/territories as the number of bidders decreases, prices continuously fall, albeit with different magnitudes. While in other countries/territories the decline in prices is particularly concentrated at the lower end of the scale, that is when bidder numbers increase from 1 to 2 or 3 (Fazekas & Kocsis, 2017; Andrey Yakovlev et al., 2014). Interestingly, at the upper end of the distribution, at 4 or 5+ bidders, prices start to increase in some countries/territories.

> 3 Please note that for these interacted regressions, we simplified the buyer location control variable from region to country/territory (only 8 categories). This makes the interactions tractable and easily visualizable, albeit R2 decreases somewhat.

26
<!-- Start of picture text --> Predictive Margins<br>oo<br>2<br>a |<br>= ———<br>mm OT<br>on<br>a<br>ink}<br>=<br>a<br>fe<br>as tm eee ee<br>7 re<br>| 2 4 4 5-0 9-75<br>bidder number categories<br><!-- End of picture text -->
<!-- Start of picture text --> Predictive Margins<br>°<br>foe)<br>8<br>s<br>= a ico} ee<br>Do<br>Ke)<br>mo]<br>2<br>no]Sv ee eo ———l ND Ore<br>2<br>a<br>" OO<br>oO<br>1 2 3 4 5 6 7 8 9 10 11 12<br>month<br><!-- End of picture text -->
<!-- Start of picture text --> 29<br>@<br>ae)<br>_<br>ioO<br>c<br>a<br>m<br>O21<br>a<br>®<br>=<br>Goi?) _L<br>cit]<br>_<br>oO 20 \/<br>lll to |<br>0 20 40 60<br>number of bids received<br><!-- End of picture text -->

<!-- Start of picture text --> 24<br>S<br>222<br>oa<br>_<br>7<br>=<br>mM<br>29<br><2]<br>@<br>o<br>YL<br><2]<br>@<br>o 1.46<br>16 | | | | | | |<br>0.00 0.25 0.50 0.75 1.00<br>supplier annual market share<br><!-- End of picture text -->
As we have developed a range of different models, we have to select the best one for the savings calculations following the criteria outlined in the methods section (prediction accuracy on the test data set and model interpretability). We used the most complete model from each type of model developed, simple OLS, interacted OLS, and Random Forest. Model prediction accuracy is summarized in Table 6 using R-squared and Mean Squared Error metrics. While the results are not unequivocal, the most interpretable model, the simple OLS regression, performs almost as good as the 2 others suggesting that it represents a balanced trade-off between precision and interpretability. Hence, this will be used for the subsequent savings estimations.

Table 6. Model comparison table, test data set

|Model|R2|MSE|
|---|---|---|
|OLS (Table 5, model 3)|0.886|2.612|
|Interacted OLS (Table B1, Model 3)|0.887|2.401|
|Random Forest|0.847|1.451|

_Note: Boosting was also estimated but no significant improvement over Random Forest was found._

# 5.3 Savings strategies and price impacts

Based on the prior advanced statistical models, it is possible to estimate hypothetical unit prices under alternative policy scenarios which reflect how much the government would have paid for the particular product had it used a more cost-effective procurement strategy. The basis of such estimation is the standard OLS selected above which model expresses the associations between various policy relevant factors and unit prices. In addition, we define realistic yet decisive policy changes to be used in the savings estimates. We start by outlining these suggested improvements to procurement policy (Table 7), then we show the estimated impact in the aggregate (Figure 7).

The savings model, composing of the price regression and the savings strategies, identifies about 14% savings potential per year, across the studied whole LAC health care sector (Figure 7). Such a substantial total predicted savings is achieved by drawing on only the most readily influenceable factors and assuming only relatively achievable policy changes. None of these changes requires altering the regulatory framework, rather imply relatively small tweaks to procurement process design and implementation decisions. Such a savings potential is roughly in line with prior savings identified looking at individual countries but across a host of economic sectors (de Oliveira et al, 2019).

30
Table 7. Summary of savings strategies used for price predictions

|directly policy<br>influenceable<br>quantity of purchased<br>goods|moving 30% of the lower deciles (582< units bought) to<br>the next higher decile (583-3000 items bought)|
|---|---|
|procedure type|move 5% of items of competitive procedures to restricted<br>procedure types|
|advertisement period<br>length|move 20% of shorter advertisement periods (1-12 days)<br>to longer advertisement (13-183 days)|
|month of spending|smooth spending across for Dec, Jan and Febr by<br>reallocating 30% of items to a nearby cheaper month<br>(March)|
|product bundling|moving 5% of items in the 1st decile to the 2nd decile|
|framework agreement|moving 2% of contracts without framework agreement to<br>with framework agreement|
|organisational quality:<br>avg. decision making<br>period length (days)|moving 7,5% of tenders in the longest quintiles to the 3rd<br>quintile|
|organisational quality:<br>avg. failed tenders %|move 50% of items from the lowest half success rate<br>organisations (< 67.5%) to highest success rate<br>organisations (>67.5%)|
|indirectly policy<br>influenceable<br>bidder number|moving 13% weakly competitive items (1-2 bidders) to<br>more competitive items (2-7 bidders)|
|buyer spending<br>concentration|move 5% of items from high spending concentration<br>buyers (the top 4 highest deciles of buyers) to average<br>spending concentration buyers (5th decile)|
|market concentration|move 50% of items in the highest concentration markets<br>(the top highest decile market) to lower concentration<br>markets (8th highest decile)|
|same location|increase market share of local suppliers (i.e. same state)<br>by 10%|
|company size|decrease share of small- and medium-sized companies<br>by 20% (to the advantage of large companies)|
|supplier specialisation|moving 3% of items supplied by highly specialised<br>suppliers (1-3 lowest deciles) to average specialised<br>suppliers (4th decile)|

31
<!-- Start of picture text --> savings source<br>16.0%<br>supplier specialisation<br>14.0% same location<br>Bcompany size<br>12.0% B market concentration<br>Bf buyer spending concentration<br>10.0%<br>@ bidder number<br>B organisational quality: avg, failed<br>5.0% tenders%<br>B organisational quality: avg, decision<br>making period length (days)<br>Bf product bundling<br>6.0%<br>B month of spending<br>B framework agreement<br>40%<br>BS adverteéement period length<br>8 proceduretype<br>2.0%<br>B quantity of purchased goods<br>0.0%<br><!-- End of picture text -->
# 6. Conclusions

This study has looked at a hitherto understudied aspect of health care pricing in the Latin America and Caribbean region: procurement policies. Our novel analytical focus was enabled by a unique, high-granularity item-level public procurement database covering 7 countries and 2 territories throughout the 2012-2018 period. We analyzed the impact of a host of policy-relevant predictors on unit prices for pharmaceuticals, equipment, and medical supplies, using a range of regression and machine learning methods. Overall, our models perform exceptionally well, accounting for over 80% of price variance in unseen data. These models enabled us to draw on a set of realistic policy changes to produce expected price savings in the health sector across the region. Overall, the identified small-scale interventions are predicted to lower unit prices by about 14%.

Our approach is novel, but it has a range of strengths and weaknesses which could be taken up by future research. The strength of the analytical framework rests in its rich empirical basis, government-wide administrative data of great detail; the strong links between well-established economic theories and the relationships sought; and the diversity of analytical methods, both quantitative and qualitative, leading to similar conclusions across the board. Moreover, our experiences with applying the framework in 9 LAC countries and territories suggest that the methods are replicable and largely standard across diverse contexts. Even though regulatory, data, and market differences require a great deal of care and adaptation of the methodology.

However, the analytical framework is not without its weaknesses, some of which can be addressed in future iterations or with better data. Most fundamentally, missing and incorrect data can invalidate findings which is, luckily, a relatively limited problem in transactional e-procurement systems. Nevertheless, highly technical and seemingly minor problems can pose substantial challenges. For example, lack of sufficiently standardized units of measurement or dosage information can render unit price comparisons flawed. While the analyzed products are highly standardized across the region, important quality differences may remain in the products themselves or in the delivery schedules (e.g. delivering earlier or later can have large price implications for suppliers). A crucial aspect of quality differences, especially impacting on equipment, is that the level of the hierarchical product classification used in the analysis may average over a range of important quality differences. Furthermore, missing information on payments and contract implementation raises questions about the validity of findings given that payments may well deviate substantially from contracted values.

On a methodological level, even though our models have performed exceptionally well in predicting prices on unseen data, we cannot claim to have identified causal relationships. Without more careful identification of causal effects, by for example using quasi experimental methods, the methodology can only capture correlations and has to rely on theory and case study methodology for further support.

From a policy perspective, while savings scenarios and predictors are actionable and directly relate to policy decisions made by procurement agencies, ministries of finance or individual 33
buyers, achieving and successfully implementing savings strategies remain challenging on their own. Overcoming resistance of front-line staff or upskilling key procurement officials may be more challenging than it seems at first.

34