metadata: authors: 0: author: Oliveira,Alexandre Borges De
1: author: Fabregas Masllovet,Abdoulaye
2: author: Fazekas,Mihaly
count: Brazil
docna: 0: docna: Auction Length and Prices : Evidence from
            Random Auction Closing in Brazil
docty: Policy Research Working Paper
subtopic: Regulatory Regimes,Judicial System Reform,Legal Reform,Legislation,Public Sector Economics,Social Policy,Public Finance Decentralization and
            Poverty Reduction,Legal Products,International Trade and Trade Rules,Telecommunications Infrastructure,Health Care Services Industry,ICT Applications
teratopic: Social Development,Industry,Information and Communication Technologies,International Economics and Trade,Law and Development,Governance,Public Sector Development
abstracts: cdata!: Electronic reverse auctions are the most
            used competitive method for procurement of goods and
            non-consulting services by the Federal Government of Brazil.
            These auctions are closed randomly, which perfectly
            satisfies fairness considerations but may be suboptimal from
            an efficiency perspective. There are concerns that tenders
            are closed too early and randomness favors bidders with
            algorithmic bidding software, leading to high prices. Hence,
            this paper investigates what would happen if the random
            closing rule was replaced by another rule. The paper uses
            the complete data set of completed electronic actions in
            2015–17 comprising 112 million bids for 0.9 million items
            purchased. Exploiting the random closing rule, simple OLS
            models are run with a wide set of fixed effects as well as
            covariates capturing competition. The findings point at
            alternative strategies to optimize auction design: simple
            actions such as increasing the average and minimum length of
            the random phase can result in 2.8 and 0.6 percent price
            savings, respectively, or R$540 million and R$116 million
            per year; or more complex designs such as setting the length
            to the maximum for the random phase if there are 15 bidders
            or more can yield 2.6 percent or R$ 500 million a year in
            price savings, or doing the same if a large discount is
            placed within three minutes to closing can yield 1.1 percent
            lower prices or R$ 210 million a year in savings.
geo_reg_key: 119228,517191
repnme: repnme: Auction Length and Prices : Evidence from
            Random Auction Closing in Brazil
display_title: Auction Length and Prices : Evidence
            from Random Auction Closing in Brazil
listing_relative_url: /research/2019/04/30999928/auction-length-prices-evidence-random-auction-closing-brazil
topicv3: Trade m1326924 698,Big Data m1327333 661,Telecommunications and Broadband Access
            m1327340 286,e-Government Procurement m1327708 1880,Health m1504661 3480
publishtoextweb_dt: 2019-04-22T00:00:00Z
historic_topic: Social Development,Industry,Information and Communication Technologies,International Economics and Trade,Law and Development,Governance,Public Sector Development
no_of_pages: 23
author_keys: 0: author_key: 458354
1: author_key: 1348372
2: author_key: 1318529
model_extractions: Policy Research Working Paper 8828 Auction Length and Prices Evidence from Random Auction Closing in Brazil _Alexandre Borges de Oliveira Abdoulaye Fabregas Mihály Fazekas_ Governance Global Practice April 2019
Policy Research Working Paper 8828

# **Abstract**

Electronic reverse auctions are the most used competitive method for procurement of goods and non-consulting services by the Federal Government of Brazil. These auctions are closed randomly, which perfectly satisfies fairness considerations but may be suboptimal from an efficiency perspective. There are concerns that tenders are closed too early and randomness favors bidders with algorithmic bidding software, leading to high prices. Hence, this paper investigates what would happen if the random closing rule was replaced by another rule. The paper uses the complete data set of completed electronic actions in 2015–17 comprising 112 million bids for 0.9 million items purchased. Exploiting the random closing rule, simple OLS models are run with a wide set of fixed effects as well as covariates capturing competition. The findings point at alternative strategies to optimize auction design: simple actions such as increasing the average and minimum length of the random phase can result in 2.8 and 0.6 percent price savings, respectively, or R$540 million and R$116 million per year; or more complex designs such as setting the length to the maximum for the random phase if there are 15 bidders or more can yield 2.6 percent or R$ 500 million a year in price savings, or doing the same if a large discount is placed within three minutes to closing can yield 1.1 percent lower prices or R$ 210 million a year in savings.

This paper is a product of the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at aoliveira@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
_Auction Length and Prices: Evidence from Random Auction Closing in Brazil_ Alexandre Borges de Oliveira (World Bank), Abdoulaye Fabregas (World Bank), Mihály Fazekas (Central European University and Government Transparency Institute) JEL classification: C53, D44, C57, C55, C61 Keywords: auctions, Brazil, high frequency bidding, random closing
# 1. Introduction

Reverse auctions were introduced to the Brazilian public procurement system in 1997 and almost instantly became the favorite method of government agencies and suppliers for its speed when compared to traditional procurement methods: you could award a contract in about two weeks. The possibility of submitting electronic bids for the first time amplified the success of reverse auctions in Brazil. Not only government agencies and suppliers were happy with them, but also the public at large and oversight agencies touted its transparency, as the entire process can be followed online by anyone with a computer and internet connection. In 2018, reverse auctions were the most used competitive procurement method by federal government agencies in Brazil by far.

But, more recently, this impressive history of reverse auctions in Brazil has been threatened by the rise of high-frequency bids, which almost always lower the price by only decimals. It is widely believed that highfrequency bids are placed by algorithms, or “bots” and there are strong concerns that they are impacting not only prices in public procurement but also unsettling the playing field. Once known for breaking monopolies and oligopolies in public procurement, reverse auctions are now on the spotlight over concerns that bidders with the capability of placing bids using bots have a leg up over bidders that use labor power to place their bids. The widespread use of bots overwhelms the IT infrastructure and makes it hard for humans to put a bid through when the system must handle a high frequency of bids placed by machines.

At the core of the problem is the approach to wrap up a reverse auction in the Federal Government of Brazil: reverse auctions are closed randomly by computers, following a period of time that can vary between one second and 30 minutes. This period, known as “random phase” in Brazil, is when fierce competition for contracts take place and bidders that do not have the capability of placing bids with bots claim that they do not see their bids through among the high frequency of bids placed by bots. In addition, government agencies and the public now question the wisdom of a random closing to reverse auctions: a great deal of processes are closed at the heat of the bidding or within very few seconds, sparking concerns that the random closing might be cutting short bidding that would otherwise lower prices for the government.

Some governments in Brazil have started taking action against high-frequency bids. For instance, the government of the State of São Paulo will automatically extend the time allowed for submission of bids if one is received within three minutes prior to closing. The state also requires new bids to lower the bidder’s previous bid by a minimum amount, usually 0.5% of the previous price – this is to counter bids that lower the price by cents, just enough to get ahead of the competition.

Given the central importance of auction length, we investigate what would happen if the random closing rule was replaced by another rule? Would government agencies get lower prices under different scenarios? This paper will present both a conceptual framework and novel empirical evidence to answer these questions. We used data on all electronic reverse auctions done by the Federal Government of Brazil during 2015-2017 which encompassed over 112 million bids for 7 million purchased items.

Our research revealed that longer random phases resulted in lower prices, but the effect tapers off at the tail end of the 30-minute period. Thus, it seems that closing the auction randomly does not produce optimal prices, especially when closing within seconds. Unsurprisingly, we found that the size of the discount had a material effect on prices; more specifically, larger discounts within the last three minutes of the random phase resulted in lower prices. In addition, the size of the discounts mattered more for the final auction price than the frequency of discounts. This appears to indicate that the high-frequency bidding attributed to the use of bots did not help to achieve the lowest prices, which came when discounts were larger rather than more frequent. Further to these results, we also found that a larger number of bidders resulted in lower 2
prices, albeit with a non-linear effect. In fact, when there were many bidders in the auction, the length of the random phase had little to no effect at all on prices.

Based on these findings, some related to the length of the random closing, some to the interaction of random phase length and indices of competition such as the number of bidders, we propose a mix of policy options to maximize results in electronic reverse auctions: simple actions as increasing the average and minimum length of the random phase can result in 2.8% and 0.6% price savings respectively. More elaborate and adaptive procurement tactics, such as automatically setting the maximum time allowed for the random phase upon certain events can produce robust savings: if there are 15 bidders or more, setting the maximum length for the random phase can yield 2.6% price savings, and doing the same if a large discount is placed within three minutes to closing can yield 1.1% price savings.

The paper is organized as follows: the next section explains the debut and evolution of reverse auctions in the context of public procurement in Brazil and why it became such a darling of public procurement. Then, we describe the random closing process, which is at the heart of today’s issues with electronic reverse auctions. Next, the paper presents the main findings of a review of the literature relevant to this research. After the literature review, the paper presents details of the data available for this work followed by the methodology used in the research. Finally, the paper will wrap up with one section to present the main results followed by the conclusions and policy recommendations.

# 2. Reverse auctions in the Brazilian procurement system

The Brazilian government procurement system is characterized by three complementary sets of processes and procedures. Procurement of civil works and consulting services is primarily carried out via the twoenvelope system set by law 8.666 enacted in 1993. In addition, the “RDC”, Regime Diferenciado de Contratações or “Special Procurement Regime”, was introduced in 2011 specifically for procurement of strategic construction projects, including those related to the world cup and Olympic games, as well as key works projects in the health sector and in the “growth acceleration program” or PAC. And then, there is the reverse auction system, regulated by law 10.520 issued in 2002, which is the default method for procurement of goods and non-consulting services. The latter is the main focus of this paper. Specifically, it will address electronic reverse auctions, which is the most important mechanism for procurement of goods and non-consulting services in Brazil. Face-to-face reverse auctions are not considered in this paper, and neither is procurement of construction works of any kind.

Electronic reverse auctions are by far the most used competitive method for procurement of goods and nonconsulting services by the Federal Government of Brazil: in 2018, 95% or R$ 19.3 billion (about US$ 2.7 billion) of competitive procurement for goods and non-consulting services were awarded through electronic reverse auctions. There is also a substantial R$ 27.5 billion (about US$ 7.9 billion) of goods and nonconsulting services that was awarded using non-competitive methods, named “dispensa” and “inexegibilidade”.

Reverse auctions were first used in Brazil by Anatel, the telecommunications regulatory agency, in 1997 and its use has spread dramatically since then. Savings achieved by the early adopters prompted the federal government to adopt the use of reverse auctions by all federal agencies in 2000 and then expand its use to all state and municipal governments in 2002. Then electronic reverse auctions became mandatory for all federal government agencies in 2005, unless there was a solid justification to do a face-to-face reverse auction. Since then a large number of electronic reverse auction systems in Brazil have been implemented; besides the federal government, most states developed their own electronic reverse auction systems such as the system in São Paulo as well as some municipal governments. In addition to systems developed by 3
government administrations, private sector companies also offered government agencies electronic platforms to carry out reverse auctions, most notably the one from Banco do Brasil, which is used by several subnational governments, and the platform offered by the commodities exchange.

This paper only covers electronic reverse auctions implemented by federal agencies using the Comprasnet platform. It does not include any procurement done by state and municipal governments nor procurement done by federal agencies that do not use Comprasnet, most notably companies owned fully or partially by the government, such as Petrobrás, Banco do Brasil and others. Regulations for reverse auctions limited their use only to off-the-shelf goods and non-consulting services. Their use for procurement of construction works is expressly forbidden. An attempt by government officials to include works failed around 2004. But reverse auction regulations introduced a novelty by not limiting its use to any monetary threshold: as long as it is procurement of off-the-shelf goods or non-consulting services, reverse auctions are the default procurement method regardless of the contract value.

- **A.** The advantages of electronic reverse auctions: Speed, transaction costs, and transparency

The speed of procuring through reverse auctions is one of its most celebrated advantages over traditional procurement methods. During 2014-2016, half of the reverse auctions were completed in just 11 days and three-quarters of them in just 13 days. Even very large contracts, over R$ 30 million, which typically take longer to award, were completed in 11 days or less half of the time. Traditional competitive procurement methods in Brazil such as concorrência or open bidding, and tomada de preços (a restricted bidding) take months to be completed.

The speed of reverse auctions can be credited to a procedure named inversão das fases or “reversal of phases”, which was a major breakthrough in procurement introduced by reverse auctions. The reversal of phases significantly cut down the time spent evaluating bids as well as lowered the number of complaints, especially those which only aimed at delaying the process.

Traditional procurement methods from law 8.666 require bids to be presented in two separate envelopes. The first envelope contains the technical proposal along with legal, financial, fiscal and technical qualification documents, while the second envelope solely contains the price. Consequently, bids are evaluated in two stages. First, only the envelopes containing the technical proposal and qualifications documents are opened and assessed by government officials with the aim of determining which of the bidders are qualified to execute a given contract. This is done for all envelopes received, regardless of what the corresponding price envelope might contain (reviewing a single technical proposal is a time-consuming task as it is dense in information). Bidders that do not meet the requirements are disqualified at this first stage and their price envelopes are returned unopened. Thus, the first stage acts like a filter through which only qualified bidders—as per the requirements set forth in the bidding documents—will move forward to the second stage. Next, the price envelopes of those bidders that were successful at the first stage are opened and the qualified bidder with the lowest evaluated price is automatically awarded the contract.

Besides having to go through the entire documentation of all bids, traditional procurement methods are known to be prone to complaints, often many of them in a single process, as any bidder that is rejected will try to reinstate its bid and other bidders will put their lawyers to work with the goal of trying to eliminate as many competitors as possible before prices are known. The dynamics of evaluating documents without knowing prices contribute to a legalistic approach to bid evaluation. The combination of all these factors results in a long process to get through bid evaluation to traditional procurement methods.

4
Electronic reverse auctions changed these dynamics by reversing the phases and starting with the price. Then, only the documentation of the lowest price bidder is evaluated and reviewed. Besides the obvious element that reviewing the documentation of one bid – the winner - is faster than reviewing for all of them regardless of ranking, the number of complaints that so significantly delay a process came down substantially, as low-ranked bidders will not make an effort and spend money for a contract they are unlikely to get. Knowing prices upfront has proven to be a factor in speeding up procurement processes and lowering the number of frivolous complaints.

A second tangible advantage of reverse auctions over traditional procurement methods is that it could be implemented electronically from end-to-end. This capability was celebrated by all stakeholders: government agencies praised its quickness, suppliers liked the ability of bidding for contracts all over the country and at a lower cost, and oversight agencies and the public at large stressed the transparency of reverse auctions. The entire process can be followed in real time, online if you have a computer with internet access. Furthermore, electronic reverse auctions automatically generate and publish all relevant documents and information online.

A full electronic process can only work effectively for the kind of items purchased using reverse auctions, off-the-shelf goods and non-consulting services, as bids are simple and easy to evaluate. It is much harder, or even not possible, to achieve the same level of automation for construction works or complex equipment that requires much more careful and detailed analysis of technical documentation and understanding the cost-quality trade-offs.

Electronic reverse auctions do not require bidders to submit bid securities, that is money which is withheld by the government until contract signature as a deterrent for bidders to fail to sign a contract or to sell its place in the ranking. Bid securities are monies that are frozen for quite a while for suppliers and thereby a cost to compete for government contracts. Eliminating bid securities made it cheaper for bidders to compete for government contracts and the possibility of submitting electronic bids lowered participation costs even further, as bidders did not have to travel to submit their bids in person and did not have to put together large hard copies of bids.

# B. Auction design and the random closing set‐up

An electronic reverse auction kicks off with publication of an advertisement for bids at the Comprasnet website. Bidders will have at least eight days to prepare and submit their bids electronically through the system. At the date and time indicated in the bidding documents, an auctioneer, who is a government official certified to carry out this type of procurement, will open up the session for bids. The starting price of each bidder will be the price quoted in the bids they uploaded to the system. Bidders will have a fixed period which is set by the buyer between one and 60 minutes to lower their prices, after which a random phase kicks in to close the reverse auction. The random phase is controlled by the system and it can vary from one second to 30 minutes.

The widely held perception of the government is that the random phase is when the real bidding takes place in electronic reverses auctions for federal procurement. This is also confirmed by descriptive statistics: in one of the few existing studies conducted on Comprasnet electronic reverse auctions, Celiktemur and Szerman (2012) observe that in a typical Comprasnet auction, a bidder places on average 1.95 bids, of which 1.36 bids, representing 70% of the total, are placed in the random phase. While it is possible to argue that bidders should place their best prices during the fixed 30 minutes that precede the random phase to avoid being cut short, the more intensive bidding observed in the random phase, however, suggests that bidders ‘save’ their best offers to the random phase, probably because they want to lower their price as little 5
as possible so as to maximize their profits. Malaga et al. (2010) provide theoretical grounding to this observed bidding behavior by establishing that, in a random close setting, the bidder with the highest bidding frequency has a higher chance of winning the auction.<sup>1</sup> This assertion implies that it is therefore rational for bidders to delay bidding during the constant phase and to start bidding intensively during the random phase to maximize their probability of winning the auction.

# 3. Literature review

Economic research on electronic reverse auctions remains scant, in part because the use of this procurement method is still relatively incipient, and because detailed transactional data from existing electronic reverse auction systems are rarely available to the public. In addition, their analysis requires specific computational infrastructure given the large volumes of data involved.

Traditional auctions, on the other hand, are one of the oldest forms of economic institutions and have therefore been subject to extensive research. Auction theory, the branch of applied economics dealing with auction markets, provides some insights that are readily transposable to electronic reverse auctions given their very similar, but inverse, features. Over the past 15 years, a growing and equally relevant body of empirical work has also emerged on Internet electronic commerce (e-commerce) auctions, taking advantage of the large volume of transactional data made available by the leading e-commerce web platforms such as eBay or Amazon.

The auction closing rule constitutes an important feature of the design of any auction system and has thereby attracted attention from researchers and practitioners alike. In fact, a diversity of auction ending designs is currently used in electronic auctions in an effort to strike the right balance between maximizing auction revenue and minimizing the costs associated with long auction processes. Broadly speaking, the existing literature distinguishes three types of auction closing rules in second-price ascending (English) auctions: (i) the hard close (HC), corresponding to a predefined time limit known to all bidders at the start of the auction; (ii) the soft close (SC), which automatically extends the auction time for a predetermined amount of time when new bids are received near the end of the auction; and (iii) the random close (RC), which assigns a time limit to each auction based on a random distribution and prevents bidders from anticipating the auction closing time.

Several papers have examined the effect of the closing rule on bidders’ behavior. Cassady (1967) describes a conventional auction with a time limit controlled by an hour glass and observes that bidders wait for the sand to nearly run out before they start placing their bids. This practice, known as late bidding or “sniping”, can still be observed in modern-day Internet electronic auctions and has since then been confirmed by empirical analysis. Ockenfels and Roth (2003) analyze data from the two leading Internet auction platforms, eBay and Amazon, which use respectively a hard close (eBay) and a soft close (Amazon): they find that the difference in auction ending rules is sufficient to explain the late bidding observed in the data. This is consistent with Malaga et al. (2010), which suggest that a random close design can discourage late bidding, enabling all bidders to compete more fairly and potentially increasing the revenue of the seller.

> 1 Malaga et al. (2010) imagine a random close auction with two bidders, A and B, in which A bids with a time delay of hA and B with a time delay of hB. They posit that the probability that B wins the auction is equal to hA divided by hA plus hB. In equation form: Pr[B wins] = hA/[hA+hB].

6
Random close designs, such as the one used in Comprasnet’s e-reverse auctions, are not common in modern-day online auctions. However, they have far predated the emergence of the Internet: Cassidy (1967) provides several examples of candle auctions, a type of auctions used in Great Britain in the 17th and 18th centuries, in which the random closing time is determined by a lit candle and the auction ends when the flames die.

In Comprasnet, the auction duration is randomly determined by the system based on a uniform distribution. According to Celiktemur and Szerman (2012), the fact that the random close is rarely observed outside Comprasnet could be due in part to IBM’s patent of the system. Using transactional data from Comprasnet, Celiktemur and Szerman (2012) report interesting patterns of bidding behavior in Comprasnet’s random close auctions. They find that (i) bidders tend to defer bidding to the end of the auction; (ii) a large part of the auctions is resolved early; (iii) bid increments are usually small; and (iv) large increments are more likely to occur early in the auction. They conclude that random close designs do not necessarily prevent late bidding and consider that random close mechanisms may lead to suboptimal prices when auctions are closed in the heat of bidding. Finally, they suggest that this effect could be offset or outweighed by increased participation since randomness increases the chances of weak bidders to win the auction.

Our paper aims to complement previous work by: (i) evaluating how Comprasnet’s random close design impacts the prices of goods and services purchased by the Federal Government of Brazil and (ii) estimating the savings that could be generated if the closing rule was set differently compared to the current random setting. Building upon a framework for measuring fiscal efficiency in public procurement that has been tested and refined through implementation in eight Latin American countries, including Brazil (World Bank, 2017), we propose a model that extends the specifications used in the previous studies, using additional controls and taking into account the interactions between competition and auction length. To the best of our knowledge, this paper represents the first attempt to quantify the effects of a random close design on the outcomes of electronic reverse auctions. It is also the first estimation of the potential savings that could be generated by using big data and artificial intelligence to optimize the auction closing rule.

# A. Hypotheses

To answer the research question presented in the introduction, we test several hypotheses relating to and addressing the existing literature. This enables us to evaluate several auction ending designs and to estimate the impact of each of these variants on the prices of the goods purchased through Comprasnet’s electronic reverse auctions.

## **_H1: Auctions with a longer random phase result in lower unit prices._**

According to this premise, we expect longer random phase durations to be associated with lower prices since longer times increase the opportunities for bidders to outbid their competitors and subsequently reduce the probability of ending the auction in the heat of bidding. Conversely, auctions with short random phase are at higher risk of closing before the market clearing price (in this case the lowest price acceptable to the bidders) is reached, impeding the government from reaping all the benefits from the competition.

In order to test this first hypothesis, we examine the independent effect of the random phase duration on unit prices. This estimation of potential savings enables the pricing of the impacts of a simple variant of Comprasnet’s random close with a longer duration of the random phase.

7
# **_H2: Auctions with a longer random phase result in lower unit prices, especially when competition is strong with many bidders present._**

Under this hypothesis, we posit that the length of the random phase results in lower prices especially when competition is sufficiently strong in the auction. In a low competition setting, with few bidders participating in the auction, an extended random phase would not necessarily translate into better prices for the government.

The specification used to test this hypothesis allows us to simulate a “smart” variant of the random ending rule, whereby the duration of the random phase is determined by a specific parameter reflecting the level of competition attained in the auction, in this case the number of bidders actively taking part in the auction.

# **_H3: Auctions with longer random phase result in lower unit prices, especially when competition is intense with a large number of bids submitted._**

This hypothesis is similar to the previous one but looks at the interacted effects of the duration of the random phase and bidding intensity in terms of the number of bids submitted in the last minutes of the random phase. We consider several resolutions for the independent variable measuring the number of bids submitted: 5 minutes, 3 minutes and 30 seconds before the end of the random phase.

This variant allows us to evaluate another “smart” design of the random ending rule in which the duration of the random phase is determined by the number of bids submitted before the close of the random phase. However, if the frequency of bidding is driven by bots or algorithms, then we can expect this hypothesis not to be supported by the data.

# **_H4: Auctions with a longer random phase result in lower unit prices, especially when competition is intense with larger bid discounts._**

This hypothesis takes into consideration the potential distortion introduced by the widespread use of bots in Comprasnet’s electronic reverse auctions. Since bots are configured to conduct high frequency bidding and to outbid the previous best bid by the minimum amount allowed, we postulate that extending the random phase based solely on the number of bidders taking part in the auction may not be optimal. Instead what is crucial for auction extension is the discounts offered.

To test this hypothesis, we look at the interaction between the duration of the random phase and an aggregate reflecting the bid discount obtained in the last minutes of the random phase. Our specification allows us to simulate an auction random close design where the length of the random phase is determined by the magnitude of the bid discount observed in the last minutes of the auction.

To summarize, the 4 hypotheses we test correspond to variants of Comprasnet’s random closing rule. H1 allows to test a simple variant with a longer duration of the random phase, while the “smart” variants used to test H2, H3 and H4 can be understood as a combination between a random close and a soft close, whereby the duration of the random phase is determined by specific parameters, such as the number of bidders participating in the auction (H2), the number of bids received in the last minutes of the random phase (H3), or the bid discount registered in the last minutes of the random phase (H4). On the other hand, we chose 8
not to evaluate hard close designs since a significant body of literature already suggests that hard close endings lead to suboptimal prices.

# 4. Data

## A. Data preparation

For the purposes of our analysis, we obtained three years (2015-2017) of detailed transactional data from Comprasnet’s online reverse auction system provided by the Brazilian Ministry of Planning, Budget and Management. For each year, we were provided with the following data sets: (i) a bidder-level data set containing general information about each auction, representing a total of 7 million observations across the three years; (ii) an item-level data set containing detailed information about each bid received during the auction, representing a total of 112 million observations; and (iii) a data set holding the milestone dates and times of all the electronic reverse auctions processes carried out through Comprasnet (0.9 million purchased items).

The data preparation phase consisted of three main steps. The first step involved merging the raw data provided by the government into a single data set by linking data from the same auctions across the different tables. It also required summarizing the data to obtain a final data set in which each observation represents an auction conducted for a specific item.

The second step was to clean the data to ensure its consistency as well as the coherence of the final sample used for the analysis. This involved removing observations with inconsistent dates and times, duplicated identifiers or missing values. Following the advice of the Brazilian Ministry of Planning, Budget and Management, we also removed all the data from agencies that are not part of the SISG ( _Sistema de Serviços Gerais_ ). All agencies that are part of SISG must use Comprasnet and SIASG for all their procurement purchases, while non-SISG agencies can choose to use or not to use these systems. When the use of the Comprasnet system is voluntary we do not observe the full purchasing activity of a buying organization potentially biasing the sample.

The third step was to generate new aggregates used in our models. The main variables generated during the data preparation phase included measures of: (i) price; (ii) duration of the auction’s random phase; and (iii) competition characteristics. Several control variables reflecting market characteristics and trends were also included in the final data set.

Besides the typical challenges associated with the data cleaning process, the large volume of transactional data managed in Comprasnet also stretched the capacity of the server-based infrastructure<sup>2</sup> provided by the Brazilian Ministry of Planning, Budget and Management. Eventually, an upgrade of the R server from 2 GB to 112 GB of RAM significantly improved the computational capacity, allowing us to successfully complete all the tasks of the data preparation phase.

## B. Indicators used in the analysis

To test the four hypotheses mentioned above, we use the unit price (in reais) as our dependent variable, and the duration of the random phase (in seconds) as our main independent variable. To test the second, third

> 2 An R instance running on a remote server with 2 GB of RAM.

9
and fourth hypotheses specifically, we also examine the interaction between the duration of the random phase and several indicators reflecting competition characteristics, namely the number of bidders participating in an auction (H2), the number of bids submitted (H3) and a measure of bid discount (H4). In all variants (H1, H2, H3 and H4), we also use a number of control variables, such as the year, the quantity purchased, market characteristics, location of procuring entity and supervisory ministry. Table 1 below presents a list of the key variables used in our analysis as well as their role in each specification.

Table 1: List of key variables used in the analysis

|Variable description|Variable role|
|---|---|
|_Price_<br>Unit price of the winning bid|DV (H1, H2, H3, H4)|
|_Auction characteristics_|IV (H1, H2, H3, H4)|
|Duration of the auction random phase (in seconds)||
|_Competition characteristics_|IV (H2)|
|_- Number of bidders_||
|Number of unique bidders participating in each auction<br>Number of active bidders in the last 30 seconds of the random phase<br>_- Bidding intensity_|IV (H3)|
|Number of bids received in the last 5 minutes of the random phase<br>Number of bids received in the last 3 minutes of the random phase<br>Number of bids received in the last 30 seconds of the random phase<br>_- Bid discount_||
|Percentage discount in the random phase|IV (H4)|
|Percentage discount in the last 5 minutes of the random phase||
|Percentage discount in the last 3 minutes of the random phase<br>Percentage discount in the last 30 seconds of the random phase||
|_Controls_|Control variables|
|Quantity purchased|(H1, H2, H3, H4)|
|Year||
|Market||
|Procuring entity||
|Location of procuring entity||
|Supervisory ministry||

The final sample used in our analysis consists of 560,163 observations, with each row in the dataset representing an auction process for the purchase of an item. Table 2 presents a set of descriptive statistics about the main variables used in the analysis.

The data reveal that the Federal Government of Brazil acquires a wide range of products and services through Comprasnet’s e-reverse auctions. Between 2015 and 2017, the government purchased more than 54,000 distinct products, with unit prices ranging from less than 1 real to more than 5 million reais.

10
Table 2: Summary statistics

|Variable|N<br>|N distinct|Median|Mean|Std. Dev.|Min.|Max.|
|---|---|---|---|---|---|---|---|
|_Price_||||||||
|Unit price|560,163||20.00|1,178.25|22,635.69|0.00|5,079,000.00|
|_Duration_||||||||
|Random phase duration (in seconds)|560,163||897.00|898.64|519.44|0.00|1,800.00|
|_Competition characteristics:_<br>_- Number of bidders_||||||||
|Number of bidders per auction|560,163||6.00|7.19|5.00|1.00|74.00|
|Number of active bidders in the last 30 sec. of random phase|445,898||0.00|0.62|1.06|0.00|13.00|
|_- Bid intensity_||||||||
|Number of bids in the last 5 min. of random phase|445,898||2.00|6.76|10.56|0.00|133.00|
|Number of bids in the last 3 min. of random phase|445,898||1.00|4.15|6.65|0.00|86.00|
|_- Bid discount_||||||||
|Bid discount in the constant phase|571,867||-0.02|115.44|35,072.43|-1.00|21,631,204.67|
|Bid discount in the random phase|453,559||-0.10|38.05|11,278.45|-1.00|6,399,999.00|
|Bid discount in the last 5 min. of random phase|289,713||-0.02|53.86|13,877.97|-1.00|6,399,999.00|
|Bid discount in the last 3 min. of random phase|248,757||-0.01|9.90|4,054.49|-1.00|1,991,999.00|
|Bid discount in the last 30 sec. of random phase|146,945||0.00|0.02|3.40|-1.00|983.13|
|_Controls_||||||||
|Year|560,163|3||||||
|Quantity purchased|560163||50.00|19,193.48|3,334,774.01|1.00|1,500,000,000.00|
|Market|560,163|8,653||||||
|Procuring entity|560,163|190||||||
|Supervisory ministry|560,163|29||||||
|Location of procuring entity|560,163|27||||||

Notes: This table reports summary statistics of the main variables in the sample dataset. For numeric variables, we display the number of valid observations, the median, the mean, the standard deviation and the minimum and maximum values. The duration of the random phase is indicated in seconds. For categorical variables, we only show the number of observations and number of distinct values.

The 190 procuring entities included in our sample present volumes of purchases ranging from as low as 13,164 reais to as high as 8,2 billion reais, an additional indication of the diversity of the demand channeled through Comprasnet’s e-reverse auctions. On the supply side, the data shows that auction processes receive an average of 7 bidders, and a median of 6 bidders. The bidders represented in our sample come from Brazil’s 27 states.

Our sample includes 8,653 markets with different characteristics. The number of contracts per market varies between 1 and more than 36,000, with a median of 434, and the volume of purchase per market ranges from a few reais to more than 1,3 billion reais, with a median of 4,1 million reais. For the purposes of our analysis, we define large markets as those with more than 250 contracts or a volume of purchases superior or equal to 500,000 reais.

As shown in Figure 1, the data confirm that the duration of the auction’s random phase follows a random uniform distribution, a result consistent with the technical specifications of Comprasnet’s random close system. As expected, the average duration of the random phase is 15 minutes. In a typical auction, the last bids are received 102 seconds before the end of the constant phase, and 131 seconds before the end of the auctions’ random phase.

11
<!-- Start of picture text --> So<br>So<br>So<br>ao<br>oO<br>Oo<br>So<br>So<br>wo<br>N<br>So<br>So<br>So<br>Oo<br>a N<br>5<br>oOsa8<br>ce<br>So<br>=]<br>cS<br>Co<br>So<br>So<br>=)<br>wo<br>o<br>a<br>0 500 1000 1500<br>df$rand_length[df$rand_length < 1801]<br><!-- End of picture text -->
<!-- Start of picture text --> 0.4<br>o<br>,os<br>Ko)<br>8<br>e703<br>So<br>3<br>0.2<br>500 1000 1500<br>Seconds<br><!-- End of picture text -->
<!-- Start of picture text --> wo<br>Oo<br>+<br>o<br>x<br>oOo<br>+<br>®D<br>oo<br>x<br>oO<br>+<br>> 8<br>Cc<br>@®<br>a]<br>a<br>xc 8<br>o<br>@®<br>Tv<br>x<br>oOo<br>2<br>®D<br>N<br>Oo<br>oO<br>+<br>®D<br>°<br>-10 5 0 5 10 15<br>df$ica_item_unitprice<br><!-- End of picture text -->
such as the number of bidders qualifying for the auction are clearly exogenous<sup>3</sup> ; others such as the discounts offered in the last 3 minutes are most likely endogenous to the dependent variable (i.e. final price is determined by the amount of discounts, but these discounts are also dependent on the price level in the auction with higher prices enabling larger discounts). Under such scenario the marginal effect of the exogenous variable remains unbiased hence we can interpret it as causal (Bun-Harrison, 2014). This is because the auction length is truly random hence its effect conditional on different competitive conditions such as more or fewer bidders will still be appropriately identified.

# 6. Results

We estimate regression equation 1 above in steps, starting from the simplest models but reporting only the final, best models with the parameters used for evaluating policy scenarios. In each model, we include the same set of control variables as highlighted in Table 1: purchased quantity, fixed effects for supervisory ministry, product market, location of the buyer, and the year of auction. While omitted variable bias cannot be ruled out as a concern for the non-random predictors, we consider the high explanatory power of the models, reaching 60-63%, as a sign that this problem is relatively contained.

First, we estimate the independent effect of random phase length on unit prices with as well as without controls for competitive conditions (H1) (Table 4, models 1-4). As expected, we see a significant negative coefficient which is quite substantial in addition. For example, holding all else constant, an extra 10 minutes of auction time would result in approximately 6% price reduction across the Brazilian federal government. This linear effect is also depicted in Figure 4.

> 3 This claim rests on the assumption that initial reference prices are determined in a standard way for all auctions hence there is no way the auction price could influence bidder participation _before_ the auction. The authors’ understanding of the Brazilian federal procurement system and our informal interviews with officials at the Brazilian Ministry of Planning, Budget and Management confirm this assumption.

15
Table 4. Main regression results-OLS predicting log unitprice (each regression includes controls for purchased quantity, fixed effects for supervisory ministry, product market, location of the buyer, and the year of auction)

|=====================================================================================<br> <br>---------------------------------------<br>|======================================<br>Dependent variabl<br>--------------------------------------<br>log unitprice|====================<br>e:<br>--------------------<br>|===================<br> <br>-------------------<br>|====================<br> <br>--------------------<br>|
|---|---|---|---|---|
|(1) (2)<br>-------------------------------------------------------------------------------------|(3) (4)<br>--------------------------------------|(5)<br>--------------------|(6)<br>-------------------|(7)<br>--------------------|
|random phase length -0.0001*** -0.0001***|-0.0001*** -0.0001***|0.00002|-0.0001***|-0.0001***|
|(0.00000) (0.00000)|(0.00001) (0.00001)|(0.00002)|(0.00001)|(0.00002)|
|number of bids in the last 5 mins|0.018***||0.014***||
||(0.0003)||(0.001)||
|log % discount in last 3 mins|-0.086***|||-0.074***|
||(0.002)|||(0.004)|
|number of bidders 0.051***|0.061*** 0.092***|0.073***|0.061***|0.091***|
|(0.002)|(0.002) (0.004)|(0.004)|(0.002)|(0.004)|
|number of bidders squared -0.0004***|-0.001*** -0.002***|-0.001***|-0.001***|-0.002***|
|(0.0001)|(0.0001)(0.00000)<br>|(0.00000)|(0.00000)|(0.00000)|
|random phase length*number of bidders||-0.00002***|||
|random phase length*number of bidders sq.||0.00000***|||
|||(0.00000)|||
|random phase length*nr.of bids in last 5 mins|||0.00000***<br>(0.00000)||
|random phase length*log % discount||||-0.00002***<br>(0.00000)|
|-------------------------------------------------------------------------------------|--------------------------------------|--------------------|-------------------|--------------------|
|Observations 334,086 334,086|266,983 106,301|334,086|266,983|106,301|
|R2 0.612 0.617|0.634 0.629|0.617|0.634|0.629|
|Adjusted R2 0.612 0.617|0.633 0.627|0.617|0.633|0.627|
|Residual Std. Error 1.437 (df = 333641) 1.428 (df = 333639)<br>=====================================================================================<br>Note:|1.402 (df = 266535) 1.345 (df = 10585<br>======================================<br>|4) 1.428 (df = 33363<br>====================<br>|7) 1.401 (df = 2665<br>===================<br>*p<0|34) 1.345 (df = 1058<br>====================<br>.1; **p<0.05; ***p<0|

16
Figure 4: Marginal effect of random phase length on log unit prices, model 1 in Table 4.

<!-- Start of picture text --> Predictive Margins with 95% CIs<br>0 180 360 540 720 900 1080 1260 1440 1620 1800<br>random phase length (seconds)<br>3.2<br>3.15<br>3.1<br>log unitprice<br>3.05<br>3<br><!-- End of picture text --> Second, our specification estimates the interaction between random phase length and the number of bidders showing up for the online auction (H2) (Table 4, model 5). As the number of bidders participating in an auction is determined prior to bidding activities and the final auction price, the criticism that it is endogenous can be largely fended off. We also enter bidder number in quadratic form to account for the diminishing effect of an additional bidder, hence the interaction effects are non-linear too. In line with expectations, random phase length continues to have a significant negative and sizeable effect on prices which increases in size as the number of bidders increases, albeit at a diminishing rate. For example, a 10 minutes longer random auction phase results in 7% lower prices when the auction has 11.5 bidders (1 sd above average), compared to only 3% lower prices when there are 2.5 bidders (1 sd below average).

Third, in order to also gauge the random phase length’s interactions with bidding intensity we estimate a specification in which we use the number of bids submitted in the last 5 minutes of the auction as the main competitive characteristic indicator (H3) (Table 4, model 6). For the number of bids variable, we find a counterintuitive result, more bids predicting higher prices, which could be driven by endogeneity bias (i.e. higher prices driving more intense bidding, rather than intense bidding driving down prices). However, this result is also consistent with the story of bots submitting a high number of bids but decreasing prices only marginally driving out genuine competition. The interacted effect of the random phase length which remains well identified, changes substantially with bidding intensity. For example, a 10 minutes longer random auction phase results in 1% lower prices when the auction has about 17 bids submitted in the last 5 minutes (1 sd above average), compared to 6% lower prices when the auction has virtually no bid submitted in the last 5 minutes (1 sd under average).

17
Fourth, in order to also gauge the random phase length’s interactions with bidding intensity we also include a specification in which we use the logged percent discount offered in the last 3 minutes of the auction as the main competitive characteristic indicator (H4) (Table 4, model 7). While conceptually this specification is highly attractive, it potentially suffers from endogeneity. Nevertheless, the effect of random phase length is once again in line with expectations, significant and large. When the discounts in the last minute are 1 sd above average, a 10 minutes lengthening of the random phase results in a 7% decrease in prices, while if discounts are 1 sd below average, the same effect size shrinks to 2%. To visually demonstrate these effects, see Figure 5 below.

Figure 5: Marginal effect of random phase length on log unit prices interacted with log discounts in the last 3 minutes, model 4 in Table 4.

<!-- Start of picture text --> Predictive Margins with 95% CIs<br>0 180 360 540 720 900 1080 1260 1440 1620 1800<br>random phase length (seconds)<br>log %disc.in last 3 mins=-10 log %disc.in last 3 mins=-8<br>log %disc.in last 3 mins=-6 log %disc.in last 3 mins=-4<br>log %disc.in last 3 mins=-2 log %disc.in last 3 mins=0<br>4<br>3.5<br>log unitprice3<br>2.5<br><!-- End of picture text -->

# 7. Conclusions and policy recommendations

The main conclusion is that the current government policy of closing reverse auctions randomly does not produce optimal prices. In our models, longer random phases contribute to lower prices, suggesting that those auctions which had a very short random phase did not give bidders enough time to lower their offers. As presented earlier in this paper, the data showed an even distribution of random phase lengths, and thus a good number of auctions closing very quickly. In fact, we recommend increasing the average and minimum length of the random phase by 450 seconds (or 7.5 minutes) and that would bring savings of 2.8% and 0.6% respectively. This would represent R$ 540 million and R$ 116 million based on 2018 spending levels.

18
A second eminent conclusion is that the size of discounts during the random phase mattered more to prices than the frequency or number of bids. For instance, one bid lowering the price significantly was better than multiple bidders lowering the price by just a few cents, or just enough to get ahead on the rankings. This appears to indicate that high-frequency bidding attributed to bots does not contribute to awarding at the lowest prices. Measures such as the ones taken by the State of São Paulo that may require a minimum reduction over the bidder’s previous bid might be productive to counter high-frequency bidding when combined with other adaptive measures, such as extending the length of the random phase.

We estimate that the federal government could save 1.1% or R$ 210 million per year if the random phase was extended to its maximum every time a price reduction in the last three minutes is bigger than the 90th percentile. Again, this shows the impact of the length of the random phase on prices, which is too great to be left as random.

The number of bidders had a material impact on prices and more bidders contributed to drive prices down, albeit with a non-linear relationship. In this vein, we recommend maxing out the length of the random phase whenever there are 15 bidders or more in the auction. That tactic can produce substantial 2.6% savings or R$ 500 million based on 2018 spending.

As a benchmark, we also include here the potential savings effect of a policy, not related to auction design: demand aggregation. The quantity purchased in an auction can have an impact on prices, as buying in bulk can lower prices compared to small purchases. In another study (World Bank, 2017), we estimated that the federal government can save 8% with bulk buying of high volume, low complexity items. That amount corresponds to about R$ 4 billion based on 2018 spending value. Thus, good planning that allows for leveraging the demand of the federal government as a whole when buying high-volume items should certainly be part of a strategy to achieve lower prices in public procurement.

Table 5. below summarizes the main conclusions and corresponding policy recommendations.

Table 5. Summary of policy options and estimated savings potential

|**Variable**|**Policy option**|**Potential**<br>**savings **|**Yearly savings**<br>**(2018 baseline)**|
|---|---|---|---|
|**Length of random**|Increase average length by 450 seconds|**2.8%**|**R$ 540 million**|
|**phase**|Increase minimum length by 450 seconds|**0.6%**|**R$ 116 million**|
|**Number of bidders**|Maximum time for random phase if 15+<br>bidders|**2.6%**|**R$ 500 million**|
||Maximum time for random phase if|||
|**Size of discount**|discount in last three minutes bigger than<br>90th percentile|**1.1%**|**R$ 210 million**|
|**Buying in bulk**|Aggregating all purchases into larger<br>tenders which are below 361 units per<br>tender|**8%**|**R$ 4 billion**<br>(*not only e‐reverse<br>auctions)|

While this research delivered a range of policy‐relevant and robust results, it merely represents the first steps towards better understanding competitive behavior in online auctions and devising ways to better tap into the potential of healthy competition. As a logical next step, further research could validate 19
whether the predicted results arise when one of the above scenarios is implemented. Moreover, more broadly, a more comprehensive assessment of competitive behavior, bidder profiles, and potentially collusive behavior could be carried out based on the rich data set available for the Federal Government of Brazil with directly policy relevant insights. Finally, if the widespread use of algorithms for bidding remains a challenge, bidding activity‐based indicators could be developed for limiting excessively frequent bidding and other unwanted behaviors.

20