The Procurement Design Assessment
System: The roadmap towards itParis, 22 June 2018
Dejan Makovšek, ITF
Adrian Bridge, QUT

2

Two challenges to advancing procurement
A lack of:• Data on procurement performance• Evidence based approach (including operational data) toprocurement design

3

Contract formats are many
Publicly financed(“Traditional”)

FinancingmethodPrivatelyfinanced (“PPP”)
Contractor
required toprice anarrow
scope

ECI (Phase 1)
Deliverymodels

ECI (Phase 2)

Alliancing

Non-pricecompetition
Contractpower

Cost-share(e.g. costsplus a fee)

Design Bid Build(DBB)

Maximumguaranteed
price

Design Build(DB)Engineering,Procurement &Construction(EPC).. A wide
scope
Contractor
engaged on
lowpowered
contract

Lump sum/fixedtime (“Turnkey”)

Note – Solid lines denote predominant choice, with dashed lines representing less common but observed
options…A high
poweredcontract4… and only a little evidence on performance
Most evidence is about on cost (and time) performance of low powered
contracts for example
SourceTimeperiod1
1927-2009

Makovšek et al. 2012

Reference
Project typeestimate
Decision to build Roads
Bridges, tunnels
Decision to build Roads
Bridges, tunnels
Decision to build Roads

Area

1995-2007

278
39
37
1536

Average Cost
overrun (%)
21.2
25.3
18.9
21.7
19.19

Lundberg et al. 2011

Decision to build Roads

1997–2009

102

21.2

SwedenLee et al. 2008
Ellis et al. 2007

Decision to build Roads
Detailed design
Roads & bridges

1985-2005
1998–2006

138
1847

11.0-13.40

South Korea
USA

Odeck, 2004

Detailed designRoads

1992-1995

620

7.88

NorwayCantarelli et al. 2012c

Detailed designRoads

1980–200923-2.9Netherlands

Ellis et al., 2007
Bordat et al. 2004

Contract value
Contract value

Roads & bridges
Roads

1998–2006
1996-2001

1908599

9.36
5.6

USAUSA

Hintze and Selstead 1991

Contract valueRoads

1985–1989

110

9.2

USA

Cantarelli et al.2012b,Flyvbjerg et al. 2003

Cantarelli et al. 2012a

1980-2009

Observ.

NW Europe

Netherlands
Slovenia

5

Many opinions on how contracts and concepts
should perform…
Some examples:- Bundling design & build will reduce variation claims due to errors andomissions in design and lead to better project outcomes.
- Stronger enforcement package (e.g. performance bonds) leads to overall
better contract performance.
- In PPPs bundling DB with OM will lead to life-cycle cost optimisation.
- …

6… actually very little evidence• Just looking at on-time/on-budget performance is insufficient, a view on
end cost is necessary as well!
• E.g. evidence on superior on-time/on-budget of D&B vs DBB for transport
infrastructure potentially available, but no view on end cost.

• What about quality/service levels?
• In-house transaction cost?

• …

7

Procurement design and performance?

Service level

Procurement
designConstruction

Operations/maintenance8Towards evidence based procurement design
Optimising procurement

ITF

Optimal risk pricing requires
breaking the project down into
different activities/contracts

QUT

Interfaces between contract
choices and competition,
bundling…

9

Economics about procurement…
• Procurement, or planning the contract, is like a hurdle race with key
decision-points that need to be cleared, or optimised:- make-or-buy decision (given current
capacities or long-term prospects);

VS.

- bundling activities(bundling decision/LCC);- based on competition or collaboration.

VS.

10Contract theory increases our understanding
Planning the contract
NIE /Contract Theory

Applications“Make-or-buy decision”

Coase’s thesis on internaland general externaltransaction costs

Outcomes
Avoiding costly mistakes by government arising becauseof lack of organizational capability and/or competence•Internalization vs Externalization ofproject’s activities•

Boundaries between government andmarket firms, vis-à-vis project activities

Avoiding risk of market firm/s holding-up government(appropriating super profits on occurrence of change binthe works)•

Learning curve and economies of scale(frequency of activity)

Allocating activities to party with superior competence
and/or capability to manage risk within activity

Williamson’s Transaction
Cost Economics→ Void in NIE/contract theory(TCE)“Bundling Decision” (amongstexternalised activities)

Property Rights Theory•

Economies of scope(PRT)•

Discrete contracts with market firms

Identifying sets of design and/or construction and/or
operations and maintenance activities with potential toincentivize decisions in upstream activities to deliver
lower whole-life costs and/or more functionality/userbenefits (i.e. positive externalities)•

Avoiding bundles that increase risk of hold-up•

Avoiding bundles that create thin competition→ Void in NIE/contract theory

Principal Agent Theory(PAT)“Exchange relationship decision” (witheach market firm representing eachcontract)•

Competitive (=Discrete) vs collaborative(=relational exchange)•

Avoid mistakenly relying on a relational exchangethat increases risk of hold-up•

Avoid mistakenly relying on a discrete exchange thatincreases contract price→ partially explained by NIE/contract theory

11… adding a capabilities perspective to address incompletedexplanations from, NIE/contract theory i.e. adding Barney’s
Resource-Based Theory to advance our understanding…
Planning the contract
Capabilities Perspective

Barney’s ResourceBased Theory(RBT)

Applications

Outcomes“Make-or-buy decision”

Allocating activities to party with superior
organizational and/or product/ion
capability to manage risk within activity“Bundling Decision”

Avoiding bundles that create thincompetition•

Helping to avoid mistakenly relying on arelational exchange that increases risk of holdup; and•Helping to avoid mistakenly relying on adiscrete exchange that increases contractprice“Exchange relationship decision”

12

Infrastructure procurement in practice?
An example of an advanced economy (Australia; similar to UK):
- Step 1: Data gathering (objectives, risks; agency’s and market
sounding/capability, unique project characteristics)- Step 2: Shortlist delivery models (consider suitability of PPP, Alliancing,Managing contractor model)- Step 3: Validation (what precedents exist for this project? What does themarket think?)- Step 4: Delivery model options analysis (Which model best achieves
objectives and reduces risk?)- Step 5: Preferred delivery model (structure preferred delivery model,consider risk; approve; execute gateway review)

13

How is infrastructure procurement informed in practice?

An example of an advanced economy e.g. Melbourne’s North-East Link project, Australia(business case at: https://northeastlink.vic.gov.au/project/businesscase)14

How is infrastructure procurement informed in practice?

15

Infrastructure procurement in practice?
Road at junction of rail
involving third party works

Cut & cover tunnel“Simple” ongrade road andelevated sectionsof road

Driven tunnel

16

How is infrastructure procurement informed in practice?

17Towards PDAS
PDAS= Procurement Design Assessment System: QUT havedeveloped the core…
Step 1. Activity Analysis (splitting the project into activities/highest levelof specialization on the market)

Step 2. Make-or-Buy Analysis
Step 3. Bundling Analysis (chasing LCC optimisation)
Step 4. Exchange Relationship Analysis (what contract type/power foreach bundle)

18

The aim in bundling and make or buy
Supplier monopoly
MARKET is technically
and/or organizationallysuperior

Supplier oligopoly

GOVERNMENT is
technically and/ororganizationally superior

Competitive
neutrality

1

RBT variable/s(capacity, rarity, costly to imitate)
dominate TCE variables

234

56

TCE variables dominate RBT(specificity, frequency, uncertainty)

78

RBT variable/s (capability, rarity,costly to imitate)
dominateTCE variables

19

How it should have been done
Contract #2(Design ofremainder)

Contract #4(Constructionof remainder)

Cut & cover tunnel

Contract #1(Design of driventunnel)

Contract #3(Construction ofdriven tunnel andcut & cover tunnel)

Third party rail alignment

O&M same, in technological terms, to rest of road network and procured as part of arange of network activities and not procured as a project-based activity in this case.

Simple road-on-gradeand elevated structures

Driven tunnel

20

Pieces to be developed to get to PDAS
Aspects to be developed/expanded the current model• Consolidation of multiple contract bundles/packages using multiple risk treatment (multiple
exchange types in one contract) in conjunction with multiple sources of finance• Consideration of activities in network settings• Integrating insights from our WG Synthesis report and current model• Inputs from ex-post analysis lacking at the moment (i.e. the first challenge of procurement)

Thank you!
Dejan Makovšek, ITF
E dejan.makovšek@itf-oecd.org
Adrian Bridge, QUT
E a.bridge@qut.edu.au