Center For Energy and Environmental 

Policy Research

Black Gold and Fool’s Gold: 

Speculation in the Oil Futures 

Market*

John E. Parsons

Reprint Series Number 228

*Reprinted from Economia, Vol. 10, 

No. 2, pp. 81-116, 2010, with kind 

permission from LACEA. All rights 

reserved. 
The MIT Center for Energy and Environmental Policy Research (CEEPR) is a joint center of 

the Department of Economics, the MIT Energy Initiative, and the Alfred P. Sloan School of 

Management. The CEEPR encourages and supports policy research on topics of interest to 

the public and private sectors in the U.S. and internationally. 

The views experessed herein are those of the authors and do not necessarily refl ect those of the 

Massachusetts Institute of Technology.
Black Gold and Fool’s Gold

Speculation in the Oil Futures Market

O

n its face, nothing looks more like a classic bubble than the acceleratin

inflation of the oil price between 2003 and mid-2008, followed by 

sudden collapse in late 2008. Starting from $30 per barrel, the pric

climbed fitfully but persistently to $100 per barrel at the end of 2007 and the

shot up above $140 by July 2008, only to collapse below $40 by the end o

2008. Clearly the price spiked dramatically, but was it a bubble?

A large number of people have pointed their fingers at the growing flow o

money into financial instruments tied to the oil price. These flows, they argue

pushed the oil price up and away from its fundamental level. The bubble burs

when the general financial market collapse put an end to this dynamic. Som

who make this argument speak with blanket disapproval of “speculation” an

“speculators” because they are not part of the “real” oil business. Some g

further still and suggest that financiers specifically “manipulated” the oil mar

ket. But the theory that the oil price spike was a speculative bubble driven b

financial flows requires neither disapproval of purely financial investments i

oil nor a judgment about motives. The thing about asset bubbles is that the

arise naturally, so to speak, in any economy sophisticated enough to develo

financial assets.

Among economists there is a prevailing skepticism toward the view tha

the oil price spike was a bubble.1 They point to the fact that the underlyin

fundamentals of supply and demand changed significantly in this period

8 

JOHN E. PARSON

Parsons is with the Massachusetts Institute of Technology.

I want to thank Ramón Espinasa of the Inter-American Development Bank, who docu

mented changes in key fundamental factors over the time horizon. This includes the declinin

crude oil production in developed countries and the rapidly increasing demand from the deve

oping countries.

1. There are also economists who are not skeptical and who have argued that the oil pric

spike was clearly a bubble, including Eckaus (2008).
Economic growth was unexpectedly rapid and persistent in several developing countries, like China and India, which increased the demand for oil.

Simultaneously, the supply of oil from some key sources fell, despite the rising price, and new sources were slow to appear. The only way to equilibrate

this increasing demand and shortfall in supply was with a sharply rising price.

The argument that demand was the fundamental factor driving the price of

oil up so sharply is bolstered by the fact that so many other commodity prices

were rising dramatically at the same time. Many of these commodities are not

traded on futures exchanges and are not assets that can be the subject of a

bubble. These include various types of iron and steel and fabricated products,

as well as cement. The price of things like engineering services increased dramatically as well.2

Rising global demand was clearly driving many prices sharply upward, and

this was probably a major factor for the price of oil. The only question is whether

changes in demand and supply curves for oil account for all of the movement

in the price. Since there is no widely accepted measure of the global demand

and supply curves for oil, it is difficult for economists to clearly demonstrate

that the spike in the oil price is entirely determined by fundamentals.

A related source of skepticism among economists is the question of the

missing stockpiles of oil. This is the dog that did not bark in the mystery of

the oil price spike. If the price of oil were to be driven above its fundamental

level at which supply and demand were matched, then consumers would cut

back on use and producers would produce more. The quantity supplied would

therefore exceed the quantity demanded, and the difference would have to go

into a stockpile that someone (namely, the financial speculators) would be

holding in hopes of a higher price in the future. Although the actual facts of

supply and demand may be difficult to pin down, barrels of oil in storage are

easy to count. No such stockpiles arose throughout the 2003–08 period, ergo

financial speculation was not the cause of the oil price spike. It must have

been supply and demand.

The economist’s skepticism is generally a healthy one, but in this case, it

is too dismissive. It overlooks how paper oil markets have been transformed.

As I explain below, successful innovations in the financial industry made it

possible for paper oil to be a financial asset in a very complete way. Once that

was accomplished, a speculative bubble became possible. Oil is no different

from equities or housing in this regard.

2. I have documented the large inflation in the cost of building nuclear power plants over

this period; see Du and Parsons (2009).
The next section of this paper documents what is known and what is no

known about the growth of financial investments in oil derivatives from th

mid-1990s through 2008. The paper then applies a widely employed mode

of oil price movements to explain the dynamics of the term structure of o

futures prices. Next, I extend this model to an analysis of the returns to finan

cial investments in oil derivatives and explore the historical relationshi

between oil price movements and profits from investments in oil derivatives

This relationship changed beginning in 2003, which may have generated th

type of dynamic in beliefs that is a key feature of speculative bubbles. More

over, the evolution of the term structure since 2003 undermines the argumen

that a bubble needs to be accompanied by an accumulation of above-groun

stockpiles of oil. The final section concludes.

Assessing the Size of Financial Investments in Oil

How large are the financial investments in oil markets? Answering this ques

tion requires sorting through exactly which type of financial investments ar

to be covered. By far the largest financial investments in oil markets are mad

through equity and debt ownership of the many, many companies engaged i

oil exploration, production, refining, and marketing. A good number of thes

companies are state owned, but many are privately owned and many publicl

traded. State claims on oil revenue in the form of royalty rights and tax claim

are another major form of financial investment. However, none of these vas

forms of investment in oil are what is really meant by the question. Instead

most questioners really want to know the scale of the pure financial bets o

the price that are made through the futures market or related channels.

In the sections below, I document the growth in exchange-traded future

and options. I also explain off-exchange trading and why so little is know

about its size. Another important issue is the shifting share of trading b

hedgers and speculators and the size of a particular type of speculative activ

ity, namely, index trades. Finally, I benchmark the scale of this growing finan

cial activity against the size of investments in the physical market for oil.

Open Interest in Exchange-Traded Futures

The New York Mercantile Exchange (NYMEX) is the major exchange o

which oil futures are traded. One metric to consider is the value of the tota

open interest in its oil futures contracts. The open interest is the total numbe
of bets in play at a given point of time, with someone being on the long side

of the bet and someone else being on the short side. Total open interest rose

from over 350,000 contracts in mid-1995 to more than 1,280,000 in mid2008, or more than 10 percent per year. A contract is for 1,000 barrels of oil,

so this is a rise from 350 million barrels to more than 1.28 billion barrels.

Multiplying the open interest measured in barrels by the price of oil per barrel shows that the value of the total open interest rose from $6.2 billion in

mid-1995 to $180 billion in mid-2008. The other major futures contracts for

crude oil are offered by the Intercontinental Exchange (ICE), which runs a

copycat contract that is pegged to the NYMEX’s own contract on West Texas

Intermediate (WTI) crude oil. The ICE also offers its own futures contract on

Brent crude, a type of oil from the North Sea and the main competitor to WTI

as a benchmark for oil.3 Combined, the open interest on these contracts is

around 15 percent of the NYMEX open interest, so that the total open interest

in mid-2008 was 1.441 billion barrels, or $202 billion. Table 1 and figure 1

show the combined total of open interest in futures, summed across the three

contracts, from 1995 through 2008. The size is given in terms of both barrels

and dollar value. A few other oil futures contracts are traded on other types

of crudes and in other exchanges around the globe, but they do not increase

the amount significantly.

In addition to simple futures positions, the exchanges offer options on the

oil price. Where a futures contract is a pure linear bet on the price of oil, an

option position is a nonlinear bet. An options position can be managed to yield

a similar exposure to changes in the oil price as a futures position, and for any

given open interest in option contracts, it is possible to calculate the open

interest in futures required to maintain an equivalent exposure. The exposure

3. The ICE parent corporation is based in Atlanta, and a large portion of its energy trading

operations are run out of Atlanta. Technically, the crude oil futures contracts are a product of

the ICE’s nominally London-based ICE Futures Europe subsidiary. Therefore, both of its contracts are subject to regulation by the U.K. Financial Services Authority (FSA) and not to oversight and regulation by the CFTC, despite the location of its operations and the U.S. delivery

point for the oil on which the WTI contract is traded. The ICE’s copycat WTI futures contract

has not been subject to the same speculative position limits that the NYMEX imposes, and the

FSA did not collect and disseminate the same information about the proportion of commercial

and noncommercial traders. For a long time, the two regulatory authorities did not actively

share information about trading and positions, which gave traders opportunities to circumvent

the tighter U.S. controls. The spike in oil prices and the public debate about the role of financial speculation and lax regulation has forced both regulators and the two exchanges to revisit

the ground rules and practices, although a formal revision to the system has not yet passed the

U.S. Congress.
created in the option markets would have added another 40 percent of ope

interest to the figure in 2000 and another 52 percent in 2004, and it woul

have more than doubled total exposure in 2008. This combined position i

shown in the table and figure for select years.4

4. Based on Büyüksahin and others (2008, table 5).

TABLE 1. Exchange-Traded Open Interest in Crude Oil

Global Global U.S. 

Total futuresa Plus optionsb production reserves reserv

Million Billion Million Billion Million Million Millio

Date barrels $ barrels $ barrels barrels barre

Jun 1995 398 6.8 . . . . . . . . . . . . . . .

Dec 1995 419 7.9 . . . . . . 25,649 . . . . . .

Jun 1996 415 8.3 . . . . . . . . . . . . . . .

Dec 1996 416 10.2 . . . . . . 26,250 . . . . . .

Jun 1997 466 9.1 . . . . . . . . . . . . . . .

Dec 1997 478 8.4 . . . . . . 27,068 . . . . . .

Jun 1998 546 7.9 . . . . . . . . . . . . . . .

Dec 1998 549 6.4 . . . . . . 27,614 . . . . . .

Jun 1999 653 12.5 . . . . . . . . . . . . . . .

Dec 1999 555 13.8 . . . . . . 27,316 . . . . . .

Jun 2000 517 16.0 724 22.5 . . . . . . . . .

Dec 2000 477 12.2 . . . . . . 28,383 . . . . . .

Jun 2001 516 13.5 . . . . . . . . . . . . . . .

Dec 2001 517 10.6 . . . . . . 28,355 . . . . . .

Jun 2002 530 14.1 . . . . . . . . . . . . . . .

Dec 2002 660 20.2 . . . . . . 28,101 . . . . . .

Jun 2003 583 17.2 . . . . . . . . . . . . . . .

Dec 2003 678 21.8 . . . . . . 29,055 . . . . . .

Jun 2004 761 28.1 1,163 42.9 . . . . . . . . .

Dec 2004 733 31.8 . . . . . . 30,333 . . . . . .

Jun 2005 866 49.8 . . . . . . . . . . . . . . .

Dec 2005 895 55.2 . . . . . . 30,871 . . . . . .

Jun 2006 1,180 88.2 . . . . . . . . . . . . . . .

Dec 2006 1,400 87.2 . . . . . . 30,858 . . . . . .

Jun 2007 1,669 118.5 . . . . . . . . . . . . . . .

Dec 2007 1,549 148.2 . . . . . . 30,821 1,238,892 30,46

Jun 2008 1,441 202.5 3,068 431.2 . . . . . . . . .

Dec 2008 1,374 58.5 . . . . . . . . . . . . . . .

Source: Open interest figures from NYMEX and ICE via Bloomberg. Global production figures are from the U.S. Energy Information Admi

istration. Reserves are from the BP Statistical Review of World Energy.

Notes:

a. Futures open interest equals the sum of all contracts for the NYMEX WTI, ICE WTI, and ICE Brent crude contracts. Open interest measur

in barrels is converted to dollars using the front month contract price only.

b. The calculation to include options is based on the ratios in Büyüksahin and others (2008, table 5).
FIGURE 1. Exchange-Traded Open Interest in Crude Oila

Source: Bloomberg. 

a. Open Interest is the sum of contracts for all months for the NYMEX WTI, ICE WTI, and ICE Brent crude contracts.

A. Millions of barrels

2,000

2,500

3,000

3,500

B. Billions of dollars

300

450

0

500

1,000

1,500

0

150
Off-Exchange Derivatives

Many of the financial trades on the oil price never come to the futures mar

kets. Swap dealers can privately negotiate with customers positions tha

look in most regards just like futures or option contracts (or bundles o

futures and option contracts). The most important difference has to do wit

counterparty credit risk. The other important difference is that dealers ca

trade tailored swaps that amount to a bundle of futures and options. It i

more convenient to the customer to buy the bundle as a single package

rather than trying to trade the components themselves. Although swaps ar

financial instruments that create oil price exposure, this trading is not generall

subject to reporting to the U.S. Commodity Futures Trading Commissio

(CFTC), and data on the volume and terms of the transactions are not pub

licly released.

Therefore, the numbers shown in table 1 and figure 1 underestimate th

true size of the open interest in crude oil, since they exclude the position

that are negotiated off the exchange. The extent of the underestimatio

depends on the character of the swap dealer’s portfolio. A swap dealer is a

intermediary. It profits by enabling the trading of others. The dealer doe

not seek to take a position in oil on its own account. It tries to match buy

ers and sellers, or, through the proper management of its overall book, t

match the aggregate positions of many buyers and many sellers. A swa

dealer will have customers who wish to sell a futures position in oil an

other customers who wish to buy a futures position in oil. Insofar as thes

match, that open interest is never seen by the futures exchange and so is no

recorded in the numbers shown in table 1 or figure 1. The dealer will onl

be compelled to bring the net position to the futures exchange if it has mor

customers on one side or the other, thereby keeping its book hedged. Th

swap dealer positions that enter into the numbers shown in table 1 an

figure 1 are only this residual fraction of the dealer’s total book that 

has brought to the exchange. Unfortunately, there is no publicly availabl

information on the size of the dealer’s book that is not brought to th

exchange, so it is impossible to precisely estimate the degree to which table 

and figure 1 underestimate the true open interest including off-exchang

positions.

The Bank for International Settlements (BIS) publishes a pair of dat

series on the notional amounts outstanding and the gross market val

ues of commodity derivative contracts traded over-the-counter, includin
oil.5 In mid-2008, the notional amounts outstanding stood at over $13 trillion, and the market value of the positions was $2.2 trillion. Unfortunately,

neither the notional amount outstanding nor the market value can be equated

to open interest, so these figures cannot be compared to what is shown in

table 1.

The recent public concern about speculation driving commodity prices

forced the CFTC to produce its special staff report on commodity swap dealers and their positions.6 The CFTC used its authority to issue a special call to

all reporting dealers for information about their aggregate positions. Unfortunately, the CFTC chose to limit the report specifically to a subset of the

dealers’ activities—namely, index trading—and did not clarify how that category was defined. In doing so, it dashed hopes of developing a complete picture of dealers’ off-exchange positions and a better estimate of the total size

of the market. While the CFTC has the data, the public does not.

Hedgers and Speculators

Open interest reflects the positions of both nonfinancial and financial traders

in oil. Many bets reflect a nonfinancial trader on one side of the contract (the

hedger), with a financial trader on the other (the speculator). This is not the

only possibility, however: both sides of the bet could involve financial traders

or both nonfinancial traders. There is no simple way to decode from the

aggregate open interest figure exactly who is doing how much of what.

The CFTC maintains a database called the large trader reporting system

(LTRS), which disaggregates the positions held by participants according to

various classifications. For example, it includes manufacturers, companies

involved in agriculture and natural resources, and producers, all of which are

natural hedgers. It also includes floor brokers/traders, futures commission

merchants, introducing brokers, commodities swaps/derivatives dealers, commodity pool operators, commodity trading advisers, insurance companies,

pension funds, and so on. A few of these categories clearly describe financial

investors or speculators, while others are financial intermediaries that may

5. See BIS (2009, table 19). The term over-the-counter (OTC) is still sometimes used as a

label for this activity, but its usage is troublesome. Any number of different ways of executing

trades—such as the ICE copycat WTI contract—have been developed to work around the old

regulatory structure using different legal strategems, including exempt commercial markets

and other loopholes. The term OTC has come to signify any trading outside the regulated

exchanges, regardless of the institutional framework for the trading.

6. See CFTC (2008).
serve both financial and nonfinancial clients, and still others may clearly b

agents of purely financial investors.

The CFTC does not publicly report information at this level of detail, how

ever. Instead, it publishes a regular Commitments of Traders (COT) repor

which aggregates these various categories into two classes: commercial an

noncommercial. Originally, this classification roughly corresponded to th

natural distinction between financial and nonfinancial participants, or specu

lators and hedgers. Over time, the CFTC, under political pressures exerte

through various channels, has allowed the meaning of the two categories t

break down, so that the report is now markedly less informative. The majo

fault arises because swap dealers, who often stand merely as an intermediar

to a speculator, became exempted from position limits imposed on specula

tors and were reclassified for reporting purposes as hedgers or “commer

cials.”7 Seen from the narrow perspective of the swap dealers’ own books o

orders, the dealers are indeed hedging: on the one side, they have the posi

tion of their ultimate customer, so they come to the futures market to hedg

that by purchasing an offsetting position. Seen from the perspective of thei

role in the marketplace, however, the positions they are bringing to th

exchange are speculations. This blurring of categories has made the regula

COT report relatively uninformative precisely when financial investment i

oil has been exploding.

The CFTC sanctioned a small number of academic studies that report som

salient statistics from the disaggregated categories of the LTRS, and thes

studies provide some insight into the share of the open interest represented b

the different financial players. Figure 2 shows the share of the total open inter

est accounted for by four categories of traders at three points in time (2000

2004, and 2008). I have aggregated the various commercials into a single cat

egory, and I show three categories that may be thought of as noncommercials

swap dealer, hedge funds, and other noncommercials.8 For commercials, th

average open interest (futures plus futures-equivalent options) increase

63 percent over the eight years, while for noncommercials it increased b

nearly 600 percent. This means that the noncommercial category grew from

7. CFTC (2008) provides a brief legislative history of this change. The issue has effectivel

been reopened, and the CFTC has announced a “Concept Release on Whether to Eliminate th

Bona Fide Hedge Exemption for Certain Swap Dealers and Create a New Limited Risk Man

agement Exemption from Speculative Position Limits” (24 March 2009).

8. Following the original source (Büyüksahin and others, 2008), I use the category lab

hedge funds, which is an aggregation of several CFTC categories that are known to represen

the trades of well-known hedge funds.
F I G U R E 2 . Shares of Open Interest on All Contracts, by Type of Trader

Sourc

The w

A. Billions of barrels

Other noncommercia

Hed

Swa

Commercials (direct on excha

B P

75%

100%

000 004 2008

%

25%

50%

000 004 2008
just over 50 percent of the total open interest to 85 percent in 2008.9 In 2008

swap dealers represented 35 percent of the total open interest, while hedg

funds represented 23 percent. Other noncommercial participants grew from

only 6 percent of open interest to 16 percent.

Later I focus on the long-maturity contracts, so it is interesting to look a

the participation shares out at the long end of the curve. For this calcula

tion, the long end is defined as contracts maturing in three or more years

In the case of commercials, the average open interest in these long maturi

ties increased 72 percent over the eight years, while for noncommercials 

increased by more than 1,200 percent, double the rate of growth in all maturi

ties taken together. The noncommercial category grew from just over 50 per

cent of the total open interest to more than 90 percent in 2008. In 2008, swa

dealers represented 59 percent of the total open interest, while hedge fund

represented 24 percent. While the size of the market at these long maturitie

is small (approximately 6 percent of total open interest) the growth in th

market over these years has meant that the total open interest in these con

tracts in 2008 was approximately equal to the total open interest in the short

est contracts (0–3 months) in 2000.

Index Funds

Index funds, or index trading, are a major class of financial investment i

commodities. Index trading is an extension of traditional portfolio manage

ment. An important concept in portfolio management is wide diversificatio

of investments across a broad spectrum of assets, maximizing the combine

return while minimizing the risk through diversification. An investment i

commodity futures is just one more so-called asset class, like governmen

bonds, corporate bonds, large stocks, and small stocks.

The idea of adding commodity futures contracts to a well-diversified port

folio has been around a long time.10 However, a variety of evolutionar

changes in various institutions made it a realistic option for a large number o

investors holding a large pot of investment dollars. A major turning poin

came with the creation of the Goldman Sachs Commodity Index (GSCI) i

1991. The GSCI, which has since been sold to Standard and Poor’s, invest

9. I have aggregated the three categories of swap dealer, hedge fund, and other noncom

mercial into a subtotal for noncommercial. Figure 2 shows the breakdown into the three ele

ments. Some of the swap dealer’s business may also represent trades by commercials, so th

may overestimate the share of noncommercial trades.

10. See, for example, Greer (1978) and Bodie and Rosansky (1980).
in an array of different commodity futures spanning energy, industrial and

precious metals, and agricultural and livestock. Because the energy futures

markets are among the deepest and most liquid financial commodity markets

in the world, the index has always been heavily weighted toward energy (now

more than 67 percent), specifically crude oil (now 48 percent). Customers can

buy into the index much as they would buy into a mutual fund. Several similar competing indexes have been created, although the major alternative in

recent years has been the Dow Jones-AIG Commodity Index (DJ-AIGCI).

Index trading may be a form of passive investing, in which the investor

takes no view of whether the commodity price is too high or too low, but

merely seeks to hold exposure bought at the market rate. As with all vehicles

for passive investing, many investors bend the concept by changing the chosen weights on different commodities according to where they think a higher

or lower return is likely to be coming over the near term. Other investors may

use the channel created by the index to take an active bet on various commodities. Each dealer offers a variety of incarnations of its commodity index,

just as mutual fund companies offer a range of different fund types, and dealers compete by structuring their indexes with different weights and so forth.

The dealers market their particular strategies.

When the popular press reports the volume of money currently invested in

commodities, they are usually reporting on index money. Indeed, the reports

often specifically conjecture on the amount of money in the GSCI, in the

DJ-AIGCI, and in funds that attempt to mimic these two funds. These reports

are conjecture benchmarked against disparate tidbits of public data. There is

no authoritative source that tallies the aggregate of funds invested in index

trades. The most recently popular benchmark derives from the CFTC’s

decision in 2006 to start producing a supplement to its COT report that purports to specifically identify index trades by swap dealers in a select set of

agricultural commodities. With these data in hand, and using the known

index weighting used in the standard GSCI and DJ-AIGCI, it is possible to

back out an estimate of the index dollars invested in all of the other commodities. The methodology is described by Masters and White, who use it to estimate a January 2006 position for the GSCI of more than $47 billion and for

the DJ-AIGCI of more than $27 billion.11 Of this, the method suggests that

the GSCI held more than $21 billion in crude oil, with the DJ-AIGCI at more

than $3 billion. Combined, this represents 43 percent of the open interest on

the combined crude contracts trading on the NYMEX and ICE at the time.

11. Masters and White (2008).
Press estimates for mid-2008 put the total index commodities investment a

$400 billion.12 Assuming the same fraction of index money invested in crud

oil, this would imply approximately $130 billion, or 64 percent of the com

bined crude open interest on the NYMEX and ICE. Masters and White esti

mate over 880 million barrels of crude in indexes, or more than $120 billion.

In its staff report on commodity swap dealers and their positions, th

CFTC purports to have calculated the total index trades held by swap deal

ers at three dates: 31 December 2007, 31 March 2008, and 30 June 2008.14 It

estimates are markedly lower than those circulating in widely cited pres

reports. For example, the report estimates the total commodity index expo

sure at $200 billion and the exposure to NYMEX crude oil at $51 billion. Th

CTFC does not explain the methodology used to determine what to includ

as an index trade. The report claims to encompass both on- and off-exchang

exposures, so one might have expected the figure to be larger than other cal

culations based exclusively on exchange positions. Instead, the CFTC num

ber is markedly smaller. The CFTC’s report does not benchmark its result t

any of these widely circulated numbers based on well-documented method

ologies and public data, so the reason for the discrepancy remains a puzzle.

Indexes are only one type of product that dealers offer, and other financia

investors—such as hedge funds—can create their own customized version o

an index. Without an authoritative public data source, this element of the o

market cannot be accurately described. Analysts have only a minimal sens

of how large it is and a clear sense that it has grown significantly over time

In addition, index funds’ own reports of their methodology, combined wit

the earlier data reported, suggest that index trading has gradually moved it

exposure out to the longer end of the forward curve.

In Comparison with the Physical Oil Market

In 2007 the global production of crude oil was approximately 30.8 billio

barrels.15 The total open interest on the NYMEX and ICE was approximatel

1.5 billion barrels, or 5 percent of total annual production. For compariso

purposes, table 1 presents the total flow of oil during the calendar year an

12. Stewart Bailey, “‘Tidal Wave’ in Commodities Rises to $400 Billion,” Bloomberg, 

April 2008. Available online at www.bloomberg.com/apps/news?pid=newsarchive&sid=a8KQ

Vrvchkw.

13. Masters and White (2008).

14. CFTC (2008).

15. U.S. Energy Information Administration (www.eia.doe.gov/emeu/international/o

production.html).
the total stock of bets on oil in place at the end of the year. The table shows

that total open interest as a share of total production was fairly stable at

around 2 percent until it began to climb in 2002.

Some of those who argue that financial investments drove the oil price

spike talk as if the flow of money into futures was somehow actually in competition with physical demand for the flow of production. Masters and White,

for example, compare the flow of money into oil with the increased consumption of oil in China.16 However, financial investors do not consume oil, and

they do not in any direct way divert production away from delivery to satisfy

demand. This obvious error annoys the inner pedant of many economists, and

it takes the focus away from other channels by which this flow of funds might

be affecting price. Some of these other channels are also mentioned in Masters and White, among other popular sources.17

While the flow of funds into oil futures and related financial investments

is not in competition with demand for actual oil consumption, it is a form of

ownership of oil. It is analogous to a flow of passive equity capital into other

assets: equity in the sense of taking on the exposure to price movements; passive in the sense that the assumed exposure is exogenous and there is no direct

management of the underlying asset. It would be analogous to a flow of funds

buying up rental housing or sharing an equity stake in owner-occupied housing. The financial owner isn’t seeking to evict anyone and hold the house

empty and unused. The price of housing will not rise because of competition

with renters for scarce homes. Can the price of housing, however, be driven

up by this new flow of equity capital into the market? Can the price of oil

and oil-linked assets be driven up, in part, by this new flow of equity capital into oil?

Restating the question this way suggests that the flow of financial investments in oil should be compared not to the annual production of oil, but to the

asset base of oil. This is hard to do for a number of reasons, so I oversimplify

to arrive at an initial calculation. In 2007 the global proved reserves of crude

oil totaled approximately 1.2 trillion barrels.18 The total open interest on the

NYMEX and ICE was approximately 1.5 billion barrels, or slightly more

than one-tenth of 1 percent of total proved reserves. Including options probably doubles this fraction. Of course, the vast majority of these reserves are

controlled by nation-states or their national oil companies, and the assets are

16. Masters and White (2008).

17. Masters and White (2008).

18. U.S. Energy Information Administration (www.eia.doe.gov/emeu/international/oil

reserves.html), citing the BP Statistical Review of World Energy, June 2008.
not available for trade. Total U.S. reserves were just over 30 billion barrels, s

the total open interest equals just over 5 percent of reserves—just over 10 per

cent if options are included. These figures are included in table 1 for conve

nience. The 10 percent ratio represents the total accumulation as of year-en

2007, but half of this was accumulated over the period 2004–07. The issu

for analysis, then, becomes what would happen if a new flow of capital sud

denly purchased an equity stake in 5 percent of the oil assets. Would tha

asset demand (as opposed to consumption demand) in that window of tim

move the asset price?19

Masters and White argue that order flow in an asset market drives price.2

That is one possible channel through which this could happen, but the

rhetorically exaggerate the size and duration of order flow in asset markets

and they never attempt to take the theory to the data relevant for the oil mar

ket. They quickly revert to comparing financial flows to consumption, whic

is the wrong approach. On its own, I doubt that a flow of that size could hav

much of an impact on an asset base of that size.21 Something more woul

have to occur for this financial flow to matter very significantly. I return t

this later.

Oil Price Dynamics

A sound discussion of the returns to an investment in oil futures requires 

digression into the stochastic dynamics of oil prices and some detail on th

term structure of futures prices. An important new dynamic that arose durin

the oil price spike of 2003–08 is reflected only in an analysis of the full term

structure. Discussions that concentrate exclusively on movements in the spo

price miss this key historical fact. This new dynamic is important to how

19. Proved reserves are a widely misunderstood concept, among economists as well a

the broader populace. Proved reserves are not comparable to the well-defined stock of a non

renewable resource at the heart of Hotelling’s famous rule (Hotelling, 1931), but are mor

analogous to an inventory of manufactured goods (Adelman, 1993). The production proces

involves investment in exploration and other drilling, and the returns to the investment arriv

stochastically, as with a Poisson process. Also, the practice of determining and declarin

proved reserves varies widely across countries, so that the data from many countries are a man

aged figure tied less to underlying facts of geology and economics than to politics.

20. Masters and White (2008).

21. The CFTC, together with other agencies, did make some effort to address the order flo

question: see the Interagency Task Force on Commodity Markets (2008), which summarize

the results documented in Büyüksahin and others (2008). They are skeptical that order flo

played a significant role.
investor beliefs evolved during this time, and it is relevant to how a bubble

might have grown.

Modeling Oil Price Dynamics

The evolution of the oil price is determined by a host of factors. On the supply side, there are developments in technology that open up the possibility of

drilling in previously inaccessible locations, movements in various factor

prices, and the stochastic realization of new resource discoveries. On the

demand side, there is the growth of global population and general economic

level of development, the changing suite of technologies for fueling transportation, and so on. Many political factors are also at work on both the supply and the demand side. For the purpose of this paper, it is convenient to

collapse all these many factors and complicated dynamics into an oversimplified taxonomy that distinguishes just two sets of factors. The first set comprises short-term, transitory shocks to either supply or demand. The second

set causes a permanent shift in expectations of the long-term equilibrium

price. I speak in reduced form as if the price is directly moved either by a

short-term, transitory factor or by a long-term, permanent factor, without

expounding on the process of equilibration between the supply curve and the

demand curve.

Just such a two-factor model was originally developed by Gibson and

Schwartz and then reinterpreted by Baker, Mayfield, and Parsons and Schwartz

and Smith, with the two factors represented as a short-term, transient factor

and a long-term, lasting factor.22 Each factor is subject to random shocks.

Shocks to the short-term factor do not have a lasting effect on the price of oil

at long horizons, but effects dissipate gradually. In contrast, shocks to the

long-term factor are lasting and thus cumulative.

Despite the oversimplification involved, this two-factor model of the oil

price is a convenient tool for narrating at least some of the dynamics observed

in the actual history of the spot price of oil. Figure 3 graphs the oil price from

1986 to 2008. In the period before 2003, the price occasionally swings up or

down, but it then reverses back toward its central tendency. These swings are

a reflection of short-term, transitory disturbances to either supply or demand

(or both). Before the recent run-up in the oil price, the most dramatic swing

was the sharp price spike occasioned by the first Gulf War in late 1990. Starting from a level below $18 per barrel in mid-July, the price peaked above

22. Gibson and Schwartz (1990); Schwartz (1997); Baker, Mayfield, and Parsons (1998);

Schwartz and Smith (2000).
$40 per barrel in October, then fell back below $18 per barrel by late Februar

1991. A less dramatic drop in prices occurred in 1993, when conflicts withi

OPEC resulted in a temporary glut of supplies and prices went from ove

$18 per barrel in August to below $14 per barrel in December, recoverin

back to over $18 per barrel again in June 1994. This temporary drop in th

spot price occasioned the brush with bankruptcy by the German compan

Metallgesellschaft, as a result of its speculation in oil futures.

Distinct from these short-run, transitory price swings are the movement

that seem likely to reflect shifts in longer-term fundamentals and thus ma

have a lasting impact on price. In the two-factor model, these longer-term

shifts are represented as a random walk. Each innovation shifts the forecaste

price at all horizons. Innovations may compound, leading to a persistent ris

or fall in the price, or innovations may cancel one another out. The key is tha

the compound result is not predictable, whereas for the transitory swings th

reversal is predictable.

From December 1998 to November 2000, the price nearly tripled from

$11 per barrel to $35 per barrel. This dramatic increase was marked by sev

eral reversals of around $10 per barrel before the upward trend recovered

Then, within the space of slightly more than a year, the price fell again t

FIGURE 3. Oil Price, 1986–2008a

Source: Bloomberg. 

a. The series shown is the front-month WTI futures price, weekly from July 1986 through December 2008.

$100

$120

$140

Dollars per barrel

$0

$20

$40

$60

$80 Aug-86

Aug-87Aug-88Aug-89Aug-90Aug-91Aug-92Aug-93Aug-94Aug-95Aug-96Aug-97Aug-98Aug-99Aug-00Aug-01Aug-02Aug-03Aug-04Aug-05Aug-06Aug-07Aug-08
$18 per barrel and recovered as quickly to above $30, continuing its rise

marked by swings of as much as $10 per barrel. For all of these swings, the

recovery occurs within less than a year, sometimes within a couple of months.

During the price run-up of 2003 through mid-2008, the series still exhibits

some striking swings around the realized trend, such as the drop in price in

late 2006.

The largest apparent swing in the price is clearly the spike in mid-2008. In

this case, reading the data in terms of the two-factor model is more difficult,

albeit not impossible. The swing is too large to be attributed completely to

short-run, transitory shocks whose dissipation should have been anticipated.

Instead, believers that the spike was caused by fundamental movements in

supply and demand are forced to explain this as one of those peculiar sample

paths that arise in any random walk: at least a good portion of the price runup was expected to persist, and it might have done so had there not been a

global financial collapse and resulting economic recession. The drop, or at

least a major portion of it, was unanticipated and reflects an unusually large

shock to the long-term factor.

In this two-factor model, the observed volatility of the spot price is a function of the volatilities of both factors. Typical estimates put the total raw

annualized weekly volatility of returns on the spot oil price in the neighborhood of 30–35 percent, with approximately one-half of that volatility coming

from short-term, transitory shocks that dissipate, leaving the long-term factor

with a volatility of close to 14–16 percent.23

The Term Structure of Futures Prices

Figure 4 displays the graphs of the term structure of futures prices prevailing

at several different dates. The horizontal axis shows the time to maturity for

the futures contracts whose prices are graphed. Each solid line reflects the full

term structure prevailing on a single date. The points on one line correspond

to the prices of contracts maturing at different horizons in the future. Moving

from line to line shows how the term structure has moved between dates.

The range of prices at short maturities is much larger than the range of

prices at long maturities. If spot prices evolve according to the two-factor

model described above, the volatility of short-maturity contracts will be higher

than the volatility of long-maturity contracts. Volatility in short-term elements of supply and demand will have a large impact on short-maturity con

23. See Schwartz and Smith (2000); Herce, Parsons, and Ready (2006).
tracts, but only a small impact on long-maturity contracts. This is because th

short-term factors are transitory. There is time for the system to adjust bac

to the equilibrium level governed by the long-term factor. The impact of 

shock to the short-term factor on a long-term futures price should decreas

asymptotically to zero as the maturity of the contract goes to infinity. If th

long-term factor is allowed to evolve, there will be volatility in long-term

futures contracts, too, but volatility in the long-term factor affects both short

and long-maturity contracts equally. Hence, volatility in short-maturity con

tracts is the product of volatility in both the short-term and the long-term

factors, while volatility in the long-maturity contracts is the product solely o

volatility in the long-term factor. Short-maturity contracts are more volatile

Figure 4 can also help clarify some classic terminology employed in th

futures market. The top curve in the figure displays the shape known as back

wardation: the price is declining with maturity. The bottom curve displays th

shape known as contango: the price is rising with maturity. The oil pric

curve is sometimes in backwardation and sometimes in contango. One facto

determining whether it is in backwardation or contango is the current realiza

tion of shocks to the short-term factor. When supplies have been temporaril

disrupted and the spot price has risen, the spot price is expected to declin

again, reverting to the long-term level. If there is a temporary glut of suppl

FIGURE 4. The Term Structure of Oil Futures on Selected Dates

Source: Bloomberg. 

$30

$40

Dollars per barrel

$10

$20

0

306090

120150180210240270300330360390420450480510540570600

Maturity in days
on the market, then the spot price will clear at a temporarily low level, and

expectations will be that it will rise again to the long-term level. These expectations about the future spot price at date t + τ are translated to the current

futures price (at date t) for a contract maturing τ periods forward.

Figure 5 clearly shows this expectation component of the term structure of

futures prices. The figure shows the time series of the spot price during just

the early part of the analysis, 1986–2002. Overlaid on top are the futures

price term structures at eleven selected dates. Each term structure is placed

so that the price of a futures contract of a given maturity is aligned with the

actual date of maturity. For example, during the first Gulf War, the spot price

on 9 October 1990 was near its peak at just over $40.00 a barrel, the price for

a futures contract for delivery three months later was $35.80, and the price

for a futures contract maturing fifteen months later was $26.03. The declining term structure clearly reflected market expectations that the Gulf War

would not disrupt supply for too long and that spot prices would come back

down again. In December 1993, when spot prices had fallen to $14.67, the

price for a three-month futures contract was $15.51 and the price for a fifteenmonth contract was $17.49. This likely reflects market expectations that the

glut of supply in December would not last and the spot price would climb

FIGURE 5. Selected Futures Term Structures, 1986–2002

Source: Bloomberg. 

$30

$40

Dollars per barrel

$10

$20

Aug-86Aug-87Aug-88Aug-89Aug-90Aug-91Aug-92Aug-93Aug-94Aug-95Aug-96Aug-97Aug-98Aug-99Aug-00Aug-01Aug-02Aug-03
back up. For 1993–2002, the figure shows five futures term structures. Dur

ing this period, the price rose and fell sharply below and sharply abov

$20.00 a barrel. The five term structures charted at the various peaks an

troughs of this movement show that the market was forecasting that the pric

would return back to a more central number, perhaps somewhere in th

neighborhood of $20.00 a barrel.

Although oil futures fluctuate between backwardation and contango, o

average they have been backwardated: more often than not, the front end con

tract has been the most expensive contract, and the term structure of future

prices at a given point in time has declined with maturity, at least near th

front end of the curve. This is not surprising. The two-factor theoretica

model described above is consistent with a curve that is either more often i

backwardation or more often in contango, depending on whether the equilib

rium risk premium associated with the short-term factor is positive or nega

tive.24 This equilibrium risk-premium reflects equilibrium in the underlyin

operation of storage for the commodity, as well as how capital markets pric

this risk and other portfolio risks. As it happens, the cost of storing oil abov

ground is very high, and, at least historically, the short-term factor has pai

a positive risk premium—that is, the term structure of oil prices is usuall

backwardated. For many other commodities, the term structure is more ofte

in contango. Oil is unusual in this regard.

The Strategy and Returns to Investment in Oil Futures

How does ownership of oil futures contribute to an investment portfolio? I

the section below, I describe the two different sources of return. One sourc

is related to the shape of the term structure, while the other is related to th

oil price level. I then show that the first source of return dominated in th

early period of 1985–2002, while the second source of return dominated i

2003–08. The string of profits earned from rising oil prices in this secon

period played an important role. As experience in other asset markets shows

investors often draw strong, unfounded conclusions from a short history o

returns, and the dynamics of these beliefs can be a contributing factor to a

asset bubble.

24. The risk premium on the long-term factor, together with the expected drift of the com

modity price, determines whether the curve is rising or falling out at the long end of the term

structure. Thus, even if the term structure is falling at the front end, it can be rising at the lon

end. Historically, the slope at the long end has fluctuated tremendously.
The Returns to an Investment in Oil Futures

The model assumes that oil price dynamics are governed by two factors.

Therefore, an investment in oil futures will capture two distinct sources of

return: the short-term risk premium and the long-term risk premium. Different

portfolios of futures contain different amounts of the two risks, so they capture different mixtures of the two returns. Financial engineering or risk management is all about being able to craft any combination of returns desired.

For example, a simple long position in the one-month futures contract will

contain both the short-term and the long-term risk in approximately equal

measure (based on estimates previously cited). A long position in the twomonth futures contract will have the same amount of the long-term risk, but

a smaller amount of the short-term risk. Constructing a short position in one

contract and a long position in another contract produces a linear combination of the two risk factors, and since contracts with different maturities contain different amounts of the two risk factors, it is possible to structure this

short-long portfolio so as to own exclusively one factor or the other.25 Whatever portfolio of the two factors of risk is chosen, this will then be blended

into a larger investment portfolio where risks and returns will be combined

according to the Markowitz portfolio theory.

Marketed index funds are long investments in futures, and all of the original articles that advocated adding commodities to an investment portfolio

were written at a time when only short-maturity contracts were available.

Hence, the portfolio usually involves long positions in short maturities and

includes returns associated with both the short- and the long-term factor. Figure 6 illustrates why this is important.26 The figure shows the spot crude oil

price from 1983 to 1993, which had ups and downs, but overall declined by

53 percent. The figure also shows the returns earned on a long portfolio of

crude oil futures in the same period. Although the spot price fell, a long port

25. The German industrial company Metallgesellschaft nearly went bankrupt when a subsidiary got it wrong. It was short a set of long-maturity physical supply contracts and therefore

had a negative exposure primarily to the long-term factor. It tried to hedge this by buying shortmaturity futures contracts. This successfully hedged its exposure to the long-term factor, but

added a new exposure to the short-term factor. When the short-term factor moved, this exposure was its undoing (see Mello and Parsons, 1995). Neuberger (1999) shows how the hedge

might have been done correctly.

26. The figure is based on a classic paper advertising the benefits of investing in commodity futures authored by Wharton and Yale professors Gary Gorton and Geert Rouwenhorst,

who teamed with AIG on its development of a commodity investment vehicle (Gorton and

Rouwenhorst, 2006). The figure doesn’t appear in the paper, but is taken from a presentation by

Rouwenhorst.
folio of futures earned a positive return. This demonstrates that an investmen

in futures is not necessarily the same thing as a bet that spot prices will climb

The ten-year change in the spot price reflects the realization of the long-term

factor, which over this window of time happened to fall. But a long positio

in short-maturity oil futures captures two returns in roughly equal measure

the return on the long-term factor and the return on the short-term factor

Although the long-term factor earned a negative return in this period, th

realized returns on the short-term factor more than compensated for the loss

As noted earlier, the risk premium on the short-term factor has generally bee

positive, and a portfolio that was long the short-term factor has generally, bu

not always, earned a positive return.

The fact that either factor may earn a positive or negative return is not dis

positive of whether it should be added to an investment portfolio. The key t

that decision is its contribution to diversification together with its own return

This, in turn, depends on the correlation with other securities in the portfolio

Many commodities have a modestly negative expected return on the short

term factor—due to the contango—and are nevertheless included in the port

folio in small amounts because the diversification benefits are thought to b

high. However, many investors do not carefully consider the full portfoli

optimization model and simply chase high realized returns—and many port

folio managers and financial institutions know how to market to that instinc

FIGURE 6. Returns to a Portfolio of Futures versus the Level of the Spot Price

Source: Gorton and Rouwenhorst (2006). Figure is taken from a related presentation by Rouwenhorst.
In this connection, there are two things to note about the period prior to

2003, when the new investment vehicles for commodity futures were first

being successfully developed and an initial population of investors was learning how to participate in this market. First, most commodities were limited to

futures contracts with very short maturities. Even those, like oil, that offered

contracts out many months had very little liquidity in those longer maturities.27 The original indexes were marketed using relatively rigid formulas for

their investment strategy, and they thus limited themselves to the shortest,

most liquid contracts. Moreover, because of the historic backwardation in the

oil futures curve, the shortest-maturity contracts had always exhibited the

greatest return. This was a critical fact for marketing the indexes. Second,

from 1986 onward, the level of the spot oil price did not trend markedly one

way or the other. Therefore, during this long window of time, virtually none

of the realized return on a long position in oil futures came from increases in

the spot price—that is, none was due to the long-term factor. Instead, virtually all of the realized return was due to the short-term factor.

The Changing Expectations Dynamic

Everything flipped around as the price of oil began to rise in 2003. First, the

continually rising spot price became a very important and consistent contributor to the return on a portfolio of oil futures. Second, late in 2004 the term

structure for oil switched into a deep, long-lasting contango, causing the portfolio to lose money on the short-term factor. These two conditions created a

strikingly different dynamic within the financial industry that was selling

commodity investments. In the earlier period, being long the short-term factor was the key to turning a large profit, but now, being long the long-term

factor became the key. Previously, being long a short-maturity contract and

short a long-maturity contract was the way to maximize return, whereas later

the trick was to avoid the short-maturity contracts and be long the longmaturity contracts. As documented above, liquidity moved out into the longermaturity contracts, driven primarily by activity from financial investors.

One other thing was markedly different about the period when the price

was rising. Figure 7 is a companion to figure 5. It shows the time series of the

spot price from 2003–08. Overlaid on this are the futures term structures at

27. Prior to 1989, oil futures contracts did not extend beyond one year, and liquidity in

these longer maturities was virtually nonexistent. The NYMEX gradually extended the horizon

of available maturities, offering contracts out to three years by 1994 and then up to seven years

by 1999.
nine selected dates. In figure 5, the futures prices regularly point down o

up according to whether the spot price is in a temporary peak or trough

In figure 7, however, as the spot price moves up, even long-dated future

prices move with it. The same thing generally happens on the way down.2

Throughout this period, innovations in the oil price are innovations at th

long end of the curve, innovations in the long-term factor. The future

curve is moving up and down in a more or less parallel fashion. It is bein

driven by changes in the long end of the curve. Volatility at the long en

rises dramatically, whereas volatility at the short end stays relatively con

stant. The volatility on the front-month futures contract is 39 percent i

1995–2002 and 40 percent in 2003–08. In contrast, the volatility on a two-yea

futures contract rises from approximately 15 percent in 1995–2002 to mor

than 25 percent in 2003–08. The contribution of the short-term facto

and the long-term factor to total spot price volatility is reversed: in 1995–2002

the short-term factor was responsible for 64 percent of total volatility an

the long-term factor 36 percent; in 2003–08, the short-term factor was onl

28. The December 2008 bottom does show a rising futures curve, suggesting that investor

may have understood that the very low spot price was potentially a temporary phenomeno

related, at least in part, to the financial crisis and global recession.

FIGURE 7. Selected Futures Term Structures, 2003–08

Dollars per barrel

Source: Bloomberg. 

$120

$140

$20

$40

$60

$80

$100 Jan-03

Jan-04Jan-05Jan-06Jan-07Jan-08Jan-09
responsible for 37 percent of total volatility, with the long-term factor accounting for the remaining 63 percent.

This changed dynamic at the long end of the futures term structure is

important for addressing the issue of the missing stockpiles of oil. As usually

posed, the argument about stockpiles oversimplifies both the physical oil

market and the paper oil market. The focus on above-ground stockpiles forgets that in the oil business, the above-ground storage decision is a very short

horizon problem. Oil is very expensive to store above ground, and only a very

low level of inventory is maintained at any given time. The return to aboveground storage is determined primarily by factors related to the short-run

volatility in the spot price, together with the slope of the forward curve at short

maturities. If the entire term structure is shifted up in parallel fashion, so that

the difference between the first-month contract price and the second-month

contract price stays constant, then the calculus regarding above-ground

storage is unaffected. So long as investors believe that the long-term price

of oil is high, it makes no sense for them to change their short-run production decisions to pump an extra barrel of oil for the purpose of storing the

extra unit in tanks until the time of sale. An elevated level of the entire term

structure should not produce a growing stockpile of oil stored in aboveground tanks.

If the level of the entire term structure is too high—that is, if there is a

bubble—it will certainly distort real investment and production decisions in

the oil industry, leading to a greater and greater disconnect between the

inflated price and the realized returns. That is the nature of a speculative bubble. Sooner or later the disconnect between the inflated price and fundamentals will become apparent: the bubble will burst. In the new world of oil,

however, the disconnect will not immediately be evidenced in the form of

above-ground stockpiles of oil. The false price signal from the high level of

the futures curve will steer excess investment into developing new resources

and production capacity, but these are not as simply tallied as above-ground

storage. Some analysts have tried to retreat from pressing the case about

above-ground storage and sought instead to consider underground stocks.

Unfortunately, the data there are abysmal. Only a few countries have standardsetting bodies that seriously enforce a meaningfully consistent definition of

reserves. The vast majority of global reserves are controlled by nation-states

that exercise political discretion on the public reporting of their reserves.

Additionally, the actual economic process of developing true reserves is

extremely stochastic, which creates significant problems for identifying the

accumulation of excess reserves over such a short window of time.
Consequently, there will be no agreement that the investment flows an

anticipated capacity are too large, at least for a while. Indeed, this is one ke

factor that makes oil a candidate for a speculative bubble. If it were easy t

specify an obvious benchmark against which to measure the price, the

beliefs about the price could not so easily fly free. The gradually inflatin

bubble affects longer-term decisions, and it is only in the playing out of th

longer-term decisions that the reality of the bubble becomes apparent.

To summarize, four key events came together in 2003–08 to contribute t

the oil price bubble. First, oil futures became available at longer and longe

maturities; at these maturities the purchase of futures became a bet solely o

the direction of the spot price, without any of the return from backwardatio

and, therefore, without any direct connection to short-term storage decisions

Second, the market switched into a deep, long-lasting contango that compro

mised the return on the traditional index portfolio strategy of being long th

short-maturity contracts, driving index investors out to longer maturities

Third, the oil price level began to rise, with the term structure moving up per

sistently at all maturities, so that the major returns on a futures portfolio cam

from the rising spot price. Finally, investment vehicles for commodities ha

been growing, as the large profits from the rise in the price level fueled 

rapidly growing stream of funds in search of these profits. The price level ris

continued in a lasting fashion, validating the initial investments and encour

aging additional ones.

This dynamic in the paper oil market cannot be seen independently of wha

was going on elsewhere in the economy.29 From a financial point of view, th

oil market may have been a sideshow or a symptom of the larger euphoria

even though the consequences for things like gasoline prices were prominen

in the popular media.

Conclusions

The debate about the causes of the oil price spike have prompted calls fo

changing policies regarding speculative activity in the oil markets. Unde

consideration are two types of reforms. The first type addresses who i

allowed to trade and how much. These are provisions to restrict speculation

29. Caballero, Farhi, and Gourinchas (2008) argue that the financial collapse in other ma

kets in 2007 sent investors in search of an alternative asset—and they found it, at least in par

in oil. Their argument, therefore, focuses on only the final, most spectacular rise in the price o

oil, as being caused by this flow of financial capital.
The second type of reform calls for transparency in the marketplace, bringing all transactions into the light and under the purview of regulators.

The long-standing practice in U.S. commodity futures markets gave regulatory authorities the right to impose position limits—that is, restrictions on

the size of any individual party’s position in the marketplace. These limits

were differentiated, with bona fide hedgers oftentimes exempted. The intention of the law was to enforce the limits against all nonhedgers, including all

financial players. Unfortunately, with the rise of the OTC swap market and

the extension of this exemption to swap dealers, the traditional power of this

regulation was undermined. Restoring position limits on all nonhedgers,

including swap dealers, is a useful reform that gives regulators the powers

necessary to ensure the integrity of the market.

Although this reform is useful, it will not prevent another speculative bubble in oil. The general purpose of speculative limits is to constrain manipulation, as well as to limit the sudden rise of order flows that would disrupt an

orderly market. These are smaller and shorter-lived problems quite unlike an

asset bubble driven by the type of widespread and gradually evolving beliefs

that may have been at work in the 2003–08 oil price. Position limits, while

useful, will not be useful against an asset bubble. That is really more of a

macroeconomic problem, and it is not readily managed with microeconomic

levers at the individual exchange level.

Perhaps for this reason, some would like to carry the restriction further and

simply ban all speculative activity in the oil futures market. Speculation

should not be banned, however, just because an asset bubble is a bad thing

that is driven by speculative activity and investor beliefs. It is not a bad thing

that oil became a financial asset. Rather, it facilitates many types of real

investment and thus lowers the cost of production. The possibility of an asset

bubble is an unfortunate liability that comes with the territory. The market

should learn to be cautious about the possibility, and the polity needs to be

ready to take measures to avoid the development of bubbles, but stopping

speculation would not be a useful step along the way.

It would be useful to bring the paper oil market back into the open and subject to the supervision of the CFTC. The lack of transparency is a dangerous

thing, for all the reasons that the financial crisis put into bold relief. The oil

and other energy markets have been far too opaque over the last two decades,

and there have been bad actors lurking in the shadows. It would be wise to fix

this. Moreover, it is a terribly, terribly easy step to take. Transparency is necessary to making the market function more effectively as a real market. It is

a good that everyone but a few special interests ought to agree on.
At the same time, the opacity of the paper oil markets had little to do wit

the oil price spike. If the lack of transparency made any contribution to a bub

ble in the oil price, it was probably marginal. These two issues, too, ar

mostly orthogonal. Whether the lack of transparency contributed to the bub

ble or not, there is no good reason for it.

The idea of an asset bubble is sometimes confused with the notion tha

financial investors are somehow manipulating the price. The two need no

coincide. The beliefs driving a bubble can gain traction without there bein

any identifiable individuals behind it. There is no evidence of manipulatio

on any scale corresponding to the size of the oil price spike. Individual actor

have sought to manipulate expectations and mask their activity—the mos

recent and relevant example being the company Vitol—but there are alway

cases of manipulation and masking in futures markets. They should be pros

ecuted, and an assessment should be made to ensure that the CFTC has suffi

cient resources for performing its task in this regard. Nevertheless, the issu

of active manipulation is orthogonal to the question of an oil price bubble, a

least during the 2003–08 period.

The oil price spike of 2003–08 certainly looks like a bubble. Oil ha

become a classic financial asset, and assets are subject to bubbles. Some of th

peculiar historical dynamics of the period may have coalesced in a fashion tha

fueled the type of investor beliefs that drive bubbles. Nevertheless, it is no

easy to prove that the oil price was experiencing a bubble. The benchmark fo

the right fundamental price is even more elusive for a commodity like oil tha

it is for equities or for housing. On the one hand, that means it should be eve

easier for a bubble to arise, while on the other, it makes it harder to recognize

That said, the argument that the lack of above-ground storage proves the o

price was not a bubble has no merit and is based on an old mind-set about o

prices. The price movements of 2003–08 would not have driven an accumu

lation of above-ground storage, whether the price movements were caused b

changing fundamentals or by the foolish beliefs that drive a bubble.

The fundamental equilibrium price of oil clearly shifted up in this period

It is unfortunate that many of those arguing that the oil price was bein

moved by speculators seem to want to minimize any role for fundamenta

factors; they seem to think that but for speculators, the price of oil might b

back at $30 a barrel. This is putting one’s head in the sand. But both side

seem guilty of treating the two potential causes of the oil price run-up—

namely, fundamentals and financial speculation—as if they were mutuall

exclusive. The global macroeconomy has just experienced a period of larg

financial imbalances and funds looking for a home. Housing and real-estat
prices appreciated significantly in several countries, an appreciation that is

widely, though not universally, accepted to have been a bubble. As I show in

this paper, it was precisely at this time that the oil markets were becoming

widely available as a financial asset. Through an accident of historical coincidence, the rising fundamentals produced exactly the run of high returns that

often drive irrational beliefs that a given investment can only go up, as was

the case in many housing and real-estate markets. This is the dynamic of

beliefs that underlies bubbles. I do not argue that the oil price spike was

exclusively caused by financial speculation, but financial speculation is part

of the story. This dynamic of the financial speculation is also connected to the

history of fundamentals. This interplay has been overlooked in the discussion

to date.
Comment

Ramón Espinasa: John Parsons’s paper underplays the importance of th

market forces underlying price evolution over the last decade. The huge devel

opment of the futures market that Parsons so well describes and analyzes ma

have amplified price oscillations since 2003, but such movements originate

in real market imbalances. It was not just a financial bubble: it had founda

tions in the real market.

Prices remained remarkably stable in real terms in the fifteen years betwee

1986 and 2001, as growth in world demand was met through several sources

There was a sustained increase in supply from members of the Organizatio

for Economic Cooperation and Development (OECD), particularly from th

North Sea and Mexico; a steady increase in supply from non-OECD, non

OPEC countries; and, above all, an expansion of production from th

Organization of Petroleum-Exporting Countries (OPEC), by making use o

the excess capacity in place since their failed attempt to defend prices by cut

ting down production in the first half of the 1980s. The real oil price increase

just 0.5 percent a year, on average, between 1986 and 2001, while world o

demand grew 1.6 percent a year in the period, with very similar rates for th

OECD and non-OECD countries. The increase in demand was reflected in 

growth of output of 0.7 percent a year among OECD countries and 1.1 percen

a year among non-OECD, non-OPEC countries. However, the bulk of suppl

came from the OPEC countries, which increased their production by 3.3 per

cent a year in the period.

Prices accelerated sharply between 2002 and 2005 because of a sudde

increase in world oil demand coupled with a sharp unexpected drop in sup

ply in the OECD countries, particularly in the North Sea, and in some OPEC

countries. Real prices increased by 26 percent a year between 2002 and 2005

The considerable upward pressure on prices stemmed from an increase i

world demand, which grew 2.1 percent a year. In particular, acceleratin

11
growth in the non-OECD countries, most notably in Asia, caused their

demand for oil to increase 3.9 percent a year, more than twice as fast as in

the previous fifteen years. However the pressure on prices came mainly

from unexpected shortfalls in supply. First, a sharp drop in production in the

North Sea basin translated into a drop in overall OECD supply of 1.7 percent a year at the time when world demand was accelerating. Second, Iraq

and Venezuela shut down their production in 2002–03. This put additional

pressure on the rest of OPEC, which was very rapidly approaching full

capacity utilization. Thus, the sharp surge and the shift in demand toward the

Far East in 2002–03 were met by an unexpected drop in supply from three

large exporting areas: the Caribbean, the Persian Gulf, and the North Sea.

This undoubtedly created an environment for speculative action, based on

real imbalances.

Finally, prices began falling with demand in 2006, and OPEC cut production to shore up prices. Demand then surged throughout 2007, while production dropped sharply and unexpectedly in Mexico. This recreated the

conditions for a huge, though short-lived, surge in prices, no doubt fueled by

speculative financial capital. Prices collapsed in the second half of 2008, as

OPEC reacted to the surge and the world economy entered recession following the financial crisis.

Perhaps one of the most forceful arguments against the speculative financial capital explanation of the oil price surge in 2003–08 is the behavior of

inventories and prices. If it was a bubble, inventories should grow regardless

of short-term price fluctuations.

This was not the case. Figure 8 shows the monthly year-on-year growth

rate of both prices and inventories. The two growth rates display a clear

inverse correlation. This very much argues in favor of prices being moved

by changes in real imbalances reflected in inventory changes and not by

speculative action. To explore this graphical evidence further, I estimated

controlled correlations. Results are presented in tables 2 and 3. The results

show a very strong and significant negative correlation between prices and

inventories. Table 2 shows the correlation from the market perspective,

with prices as the dependent variable. Price changes correlate inversely to

changes in inventories. The correlation is particularly strong in the period

of fastest price growth, 2002–05. Table 3 shows the correlation from the

perspective of the inventory holder, with inventories as the dependent variable. If the oil price spike was a speculative bubble, there should not be

such a strong negative correlation between inventories and prices.
FIGURE 8. Monthly Year-on-Year Variation of U.S. Petroleum Stocks and WTI Nominal Price

Source: U.S. Energy Information Administration. 

50

100

150

200

0

5

10

15

20

g_stock g_price

-10

-50

0

50

100

150

200

-20

-15

-10

-5

Jan-87 Oct-87

Jul-88

Apr-89 Jan-90 Oct-90 Jul-91

Apr-92 Jan-93 Oct-93 Jul-94

Apr-95 Jan-96 Oct-96 Jul-97

Apr-98 Jan-99 Oct-99 Jul-00

Apr-01 Jan-02 Oct-02 Jul-03

Apr-04 Jan-05 Oct-05 Jul-06

Apr-07 Jan-08 Oct-08

Jul-09

TABLE 2. Correlation of the Oil Price and U.S. Petroleum Stocks

(1) (2) (3) (4)

Explanatory variable 1986–2009 1987–2001 2002–05 2006–09

Inventories −0.684*** −1.821*** −0.522** −5.990**

(0.13) (0.18) (0.23) (0.58)

Constant 0.000 −23.424 21.938* 45.144**

(1.18) (21.31) (12.45) (9.50)

Summary statistic

No. observations 1,265 778 207 208

Rsquared 0.01 0.76 0.92 0.79

* p < 0.10.

** p < 0.05.

*** p < 0.01.

a. The dependent variable is the real oil price. Variables are expressed in logs and are demeaned. Estimation includes weekly, month

and yearly controls. Robust standard errors are in parentheses.
TABLE 3. Correlation of the Oil Price and U.S. Petroleum Stocks

(1) (2) (3) (4)

Explanatory variable 1986–2009 1987–2001 2002–05 2006–09

L.rpo −0.021*** −0.101*** −0.073** −0.102***

(0.00) (0.01) (0.03) (0.01)

Constant 0.020 10.018*** −7.916*** 6.195***

(0.20) (1.41) (2.28) (1.43)

Summary statistic

No. observations 1,260 776 206 208

Rsquared 0.01 0.85 0.79 0.92

** p < 0.05.

*** p < 0.01.

a. The dependent variable is U.S. crude oil stocks. Variables are expressed in logs and are demeaned. Estimation includes weekly,

monthly, and yearly controls. Robust standard errors are in parentheses.

To end, an aspect worth studying in future research is how the amplitude

of the price oscillations in reaction to changes in fundamentals has increased

with the huge growth of the futures market over the last decade.
References

Adelman, Morris A. 1993. The Economics of Petroleum Supply. MIT Press.

Baker, Malcolm P., E. Scott Mayfield, and John E. Parsons. 1998. “Alternative Mod

els of Uncertain Commodity Prices for Use with Modern Asset Pricing Methods

Energy Journal 19(1): 115–48.

BIS (Bank for International Settlements). 2009. Semiannual OTC Derivatives Statis

tics at End-June 2008. Basel.

Bodie, Zvi, and Victor Rosansky. 1980. “Risk and Return in Commodity Futures

Financial Analysts Journal 36(3): 27–39.

Büyüksahin, Bahattin, and others. 2008. “Fundamentals, Trader Activity, and Deriv

ative Pricing.” Working Paper. Washington: U.S. Commodity Futures Tradin

Commission.

Caballero, Ricardo J., Emmanuel Farhi, and Pierre-Olivier Gourinchas. 2008

“Financial Crash, Commodity Prices, and Global Imbalances.” BPEA 2: 1–55.

CFTC (U.S. Commodity Futures Trading Commission). 2008. Staff Report on Com

modity Swap Dealers and Index Traders with Commission Recommendation

Washington.

Du, Yangbo, and John E. Parsons. 2009. “Update on the Cost of Nuclear Power.

Working Paper 09-003. Massachusetts Institute of Technology, Center for Energ

and Environmental Policy Research.

Eckaus, Richard. 2008. “The Oil Price Really Is a Speculative Bubble.” Workin

Paper 08-007. Massachusetts Institute of Technology, Center for Energy an

Environmental Policy Research.

Gibson, Rajna, and Eduardo S. Schwartz. 1990. “Stochastic Convenience Yield an

the Pricing of Oil Contingent Claims.” Journal of Finance 45(3): 959–76.

Gorton, Gary, and K. Geert Rouwenhorst. 2006. “Facts and Fantasies about Com

modity Futures.” Financial Analysts Journal 62(2): 47–68.

Greer, Robert J. 1978. “Conservative Commodities: A Key Inflation Hedge.” Jour

nal of Portfolio Management 4(4): 26–29.

Herce, Miguel, John E. Parsons, and Robert C. Ready. 2006. “Using Futures Price

to Filter Short-Term Volatility and Recover a Latent, Long-Term Price Series fo

Oil.” Working Paper 06-005. Massachusetts Institute of Technology, Center fo

Energy and Environmental Policy Research.

Hotelling, Harold. 1931. “The Economics of Exhaustible Resources.” Journal o

Political Economy 39(2): 137–75.

Interagency Task Force on Commodity Markets. 2008. Interim Report on Crude Oi

Washington.

Masters, Michael W., and Adam K. White. 2008. “The Accidental Hunt Brother

How Institutional Investors Are Driving up Food and Energy Prices.” Availabl

online at www.accidentalhuntbrothers.com.
Mello, Antonio S., and John E. Parsons. 1995. “The Maturity Structure of a Hedge

Matters: Lessons from the Metallgesellschaft Debacle.” Journal of Applied Corporate Finance 8(1): 106–20.

Neuberger, Anthony. 1999. “Hedging Long-Term Exposures with Multiple ShortTerm Futures Contracts.” Review of Financial Studies 12(3): 429–59.

Schwartz, Eduardo S. 1997. “The Stochastic Behavior of Commodity Prices: Implications for Valuation and Hedging.” Journal of Finance 52(3): 923–73.

Schwartz, Eduardo S., and James E. Smith. 2000. “Short-Term Variations and LongTerm Dynamics in Commodity Prices.” Management Science 46(7): 893–911.
MIT CENTER FOR ENERGY AND ENVIRONMENTAL POLICY RESEARCH 

REPRINT SERIES

CEEPR Reprints are available free of charge (limited quantities.) Order online at 

ceepr@mit.edu

218 Epilogue: Post-Kyoto International 

Climate Policy, Richard Schmalensee, 

in Post-Kyoto International Climate 

Policy, J. Aldy and R. Stavins (Editors), 

Cambridge University Press, pp. 889- 

899, (2010) 

219 Winter Heating or Clean Air? 

Unintended Impacts of China's Huai 

River Policy, Douglas Almond, Yuyu 

Chen, Michael Greenstone and 

Hongbin Li, American Economic 

Review, Vol. 99, No. 2, pp. 184–190, 

(2009) 

220 Second-Best Instruments for 

Near-Term Climate Policy: Intensity 

Targets vs. the Safety Valve, 

Mort Webster, Ian Sue Wing and 

Lisa Jakobovits, Journal of 

Environmental Economics and 

Management, Vol. 59, No. 3, pp. 250–

259, (2010) 

221 Market Power in Pollution Permit 

Markets, Juan-Pablo Montero, The 

Energy Journal, Climate Change 

Policies After 2012, Vol. 30 (Special 

Issue 2), pp. 1-28, (2009) 

222 The European Union Emissions Trading 

Scheme: Origins, Allocation, and Early 

Results, A.D. Ellerman and 

B.K. Buchner, Review of Environmental 

Economics and Policy, Vol. 1, No. 1, 

pp. 66-87, (2007) 

223 Over-Allocation or Abatement? A 

Preliminary Analysis of the EU ETS 

Based on the 2005 Emissions Data, 

A.D. Ellerman and B.K. Buchner, 

Environmental and Resource 

Economics, Vol. 41, No. 2, pp. 267-287, 

(2008) 

224 The European Carbon Market in 

Action: Lessons from the First Trading 

Period, Frank Convery, D. Ellerman and 

Christian de Perthuis, Journal for 

European Environmental & Planning 

Law, Vol. 5, No. 2, pp. 215–233, (2008)

225 Cost and U.S. Public Policy for New 

Coal Power Plants with Carbon 

Capture and Sequestration, Michael R. 

Hamilton, Howard J. Herzog and John 

E. Parsons, Energy Procedia, Vol. 1, 

No. 1, pp. 4487-4494, (2009)

226 Quasi-Experimental and Experimental 

Approaches to Environmental 

Economics, Michael Greenstone and 

Ted Gayer, Journal of Environmental 

Economics and Management, Vol. 57, 

No. 1, pp. 21-44, (2008)

227 Does Air Quality Matter? Evidence 

from the Housing Market, Kenneth Y. 

Chay and Michael Greenstone, Journal 

of Political Economy, Vol. 113, No. 2, 

pp. 376-424, (2005)

228 Black Gold and Fool’s Gold: 

Speculation in the Oil Futures Market, 

John E. Parsons, Economia, Vol. 10, No. 

2, pp. 81-116, (2010)
Cambridge, Massachusetts 02142

400 Main Street (E19-411)Center for Energy and Environmental Policy ResearchMassachusetts Institute of Technology