Financial Energy Markets and the Bubble in Energy Prices:

Does the Increase in Energy Trading By Index

And Hedge Funds Affect Energy Prices?

Testimony Before a Joint Heating of the U.S. Senate Permanent
Subcommittee on Investigations of the Committee on Homeland Security and

The Governmental Affairs and the Subcommittee on Energy of the

Committee on Energy and Natural Resources

December 11,2007

By

Edward N. Krapels

Special Advisor

Financial Energy Markets

Energy Security Analysis, Inc.

Wakefield, Massachusetts

**[BACKGROUND: THE "FINANCIALIZATION" OF ENERGY MARKETS]{.underline}**

Do financial energy markets affect the level and the volatility of oil
and gas prices? We use the term "financial energy markets" here to mean
the collective of trading arenas in which forward energy prices evolve
from trades on (1) formal traditional exchanges (notably the New York
Mercantile Exchange), (2) new forms of exchanges that combine
traditional and over-the-counter transactions (notably, the
Intercontinental Exchange), and (3) bilateral energy contracts whose
prices are indexed to those of the exchanges.

Discussions within the oil and finance community reflect various
perspectives on this issue. The discussions raise a very important
question: did the increase in oil prices to almost \$100/barrel and
natural gas prices above \$10 per MMBTU in 2006 and 2007 reflect classic
commodity "bubbles" in which financial markets played a distinct, *sui
generis* role[^1]; or a "new regime" of permanently higher prices
brought about by sharp increases in demand and enduring changes in
supply, which pushed both crude oil and natural gas into suddenly much
higher marginal production costs? As always, the answers are not
mutually exclusive. We may be living in a period when there has been a
"perfect storm" of conditions conducive to higher energy prices.

This is obviously an enormously complication question. The number of
dollars involved in energy futures and over the counter markets
(collectively, the energy derivatives markets) is measured in the
hundreds of billions. The physical oil market is global in scale, and
information about global oil stocks and flows is notoriously incomplete.
The flow of investor funds into commodities; into the fuels segment of
commodities; into individual fuels; and from the long to the short side
of particular markets is also immense and has been growing rapidly in
the last five years.

The question the House and Senate committees are exploring this week is
whether the increase in the volume and open interest in oil and gas
derivatives markets has a significant impact on world crude oil and
petroleum product prices, and on U.S. natural gas prices.[^2] I believe
this is likely to be one of those questions that -- to use Gregory
Treverton's useful distinction[^3] \-- is a mystery, rather than merely
a puzzle. In their formal capacities, economists are trained to treat
problems as puzzles, amenable to rational analysis. That requires enough
information to move the problem from the mists of mystery to the
brighter lights of puzzles. There are reasons to believe that condition
does not exist, yet, in this case.

How do financial energy market activities influence energy prices? In
articles I have published on this issue[^4], I have compared the "flow
of funds" of the magnitude we are seeing today to a new wave of buyers
and sellers interested in oil and gas. Could that flow have created a
"bubble" in oil and gas prices in 2005, 2006 and 2007? Examples of such
bubbles abound. From Dutch tulip markets in the 1600s to Internet
equities in the 2000s and the subprime mortgage crisis today, asset
classes routinely go through booms and busts created -- not by any
change in the costs of production or technological change in the value
added by consumption -- but purely by virtue of a change in investors'
desire to own the asset.

There were forward and derivative instruments in oil markets as far back
as the 1860s, but they were not as ubiquitous and as easy to use as
those available today. Before the advent of modern financial markets,
the desire to own oil could manifest itself in only limited ways. One
could hoard physical barrels of oil, put them in storage, and sell them
at a later date (at a profit or a loss). Or, one could buy the equity or
debt instruments of oil producing companies.

Beginning in the 1980s, the emergence of a viable and liquid futures
market for oil made it much easier for investors and traders to deal in
the commodity: they could buy or sell contracts, settled by an Exchange.
U.S. natural gas followed suit in the early 1990s. Like any other
futures market, the oil and gas futures markets allow one class of
participants to hedge, and another class of participants to speculate.
Speculators play an important role: they allow hedgers to put aside the
risk of commodity price fluctuations to others better able or more
willing to live with them. Oil and gas producers and consumers are
hedgers, small traders and larger financial institutions, like hedge and
private equity funds, some investment banks, and specialized energy
trading outfits, are speculators.

Even though many crude oil and natural gas producers, oil refiners, and
petroleum product and natural gas consumers do not hedge, the fact
remains that New York Mercantile Exchange (NYMEX)-traded West Texas
Intermediate (WTI) crude oil and Intercontinental Exchange (ICE)
--traded Brent crude oil, American and European heating oil and
gasoline, and U.S. natural gas contracts have become benchmarks of
[both]{.underline} physical commodities and financial assets whose price
fluctuations affect the economics of the entire energy industry as well
as those buying services from that industry. Thus, even purely
commercial participants in oil and gas markets are just as affected by
the force of financial energy markets as are the speculators and hedgers
that use them every day.

Beginning in the 1990s, some participants in oil and gas markets began
to suspect that the trading behavior of institutional speculators was
influencing prices. These speculative organizations had been minor
participants in the financial oil markets since the crude oil contract
was launched in 1984. By the mid-1990s, however, the number of financial
investors trading crude oil contracts began to increase rapidly. The
increase was not confined to oil: to the contrary, one can only
understand the phenomenon, and how to deal with it, if one understands
the larger investment picture in global financial markets.

With the wide array of contracts and assorted rules on leveraging
trades, international financial markets have become extremely
complicated. In every economy, wealth is held in the form of land,
precious metals, goods, and financial instruments like stocks, bonds,
currency holdings, and futures contracts. The stock of wealth, on a
global scale, has to be tallied in the hundreds of trillions of dollars.
The largest shares are in the United States, Japan, and Western
Europe.[^5]

If the stock of global wealth can be measured in the hundreds of
trillions of dollars, the flow of funds -- which no single institution
measures systematically -- amounts to several trillions of dollars over
the course of a year. Thus, a Japanese investor may sell his real estate
in Tokyo in order to buy stocks in Malaysia, or U.S. Treasury Bills, or
crude oil futures contracts, or a trunkful of gold or silver. He may
also deposit his funds in a bank, which then makes loans, engages in
swaps, and sells futures and options in the over the counter markets.

![](media/image1.wmf)This intricate web of investments, loans, and
derivatives has grown exponentially over the last ten years. Parts of
this web are always under some pressure. There is almost always a small
meltdown or bubble somewhere. In 1998 and 1999, the meltdowns were very
large indeed. Asian equity, real estate, and currency markets collapsed.
In 2001, the meltdown occurred in U.S. and global equity markets in the
spring of 2000.

Meltdowns can happen anywhere. In late September 1998, reports began to
circulate of a successful effort by the New York Federal Reserve Bank to
orchestrate a \$3.5 billion bailout of a hedge fund (Long Term Capital).
According to new reports, "Wall Street's biggest power brokers agreed to
prop up one of their most aggressive offspring, Long-Term Capital
Management, L.P., a highflying hedge fund that was on the verge of
collapse."[^6] According to the *Wall Street Journal,* one of the "hotly
debated topics" in the meeting that reached the accord to bail out the
Fund was that its failure "would put the entire financial system at
risk" because the Long Term Capital had leveraged its several billion
dollars of investment capital into a market position that at times
exceeded \$100 billion.

**[FINANCIAL MARKETS AND ENERGY FUNDAMENTALS]{.underline}**

Some authoritative observers -- like former Federal Reserve Chairman
Alan Greenpsan and eminent oil economic Robert Mabro \-- believe the
financial markets have a *sui generis* impact on oil prices. If so,
there must be "fundamentals of paper markets" that one must assess along
with the fundamentals of the physical markets in order to obtain a
complete view of oil pricing dynamics. Others are more skeptical,
believing that futures and forward prices reflect entirely information
about the fundamentals of the physical market. Some of those who believe
the financial markets have a *sui generis* impact on prices are
advocating stricter regulation of energy trading activities. Given the
ease of international capital movements, however, it is unclear whether
regulation in and by the United States would have much effect: squeezing
one part of the energy trading balloon may only cause the bubble to
appear elsewhere.

Discussions within the oil and finance community reflect various
perspectives on the issue: do oil prices above \$50/barrel and natural
gas prices above \$5 per MMBTU reflect classic commodity "bubbles" in
which financial markets played a distinct, *sui generis* role, or a "new
regime" of permanently higher prices brought about by sharp increases in
demand, which pushed both crude oil and natural gas into suddenly much
higher marginal production costs. Recognizing that both financial and
physical dynamics are always at play, the issue nevertheless is whether
the financial dynamics have a distinct and measurable role.

The bubble argument suggests that developments in financial energy
markets (especially the increase in cash under management of hedge and
other funds, and the decisions of index-oriented funds to take long
positions in commodities, including energy) may have precipitated a
classic period of "too many buyers chasing too few sellers" of financial
oil instruments. Such periods of "excess demand" have occurred hundreds
of times in competitive markets over the course of centuries. Once oil
and gas developed futures and forward market instruments, with all of
the fungibility characteristics of such instruments, they too became
prey to purely financial bubbles. The potential for such bubbles
increased in recent years because of the massive scale of increased
involvement of financial institutions that heretofore had not been
significant players in the energy space.

For example, Robert Mabro argues that

> "Econometric models show that the net position of the so-called
> 'non-commercial traders' is correlated with the subsequent direction
> of price changes. In other words, when the non-commercial entities
> hold a net long position (they are betting on a price rise) prices
> often do rise. And the opposite impact occurs when these entities hold
> net short position. Is it not odd that the non-commercial players
> (meaning very broadly the non-oil companies) should lead and the
> commercial entities (broadly speaking oil or energy companies, oil
> users and oil-related agents) should follow in what is supposed to be
> an oil market?"[^7]

Others believe in variations of a "new regime" argument that has two
dimensions. On the supply side, they would argue that there has been a
permanent movement up the oil and gas production cost curve caused by a
lack of investment by and in the petroleum extraction industry. On the
demand side, there has been an increase in the rate of growth in oil and
gas demand (the oil side mostly from Asia; the gas side mostly from
increases in the use of combined cycle gas turbines). Taken together,
the new regime is characterized by increases in demand for oil and gas
that exceed the increase in supply. Thus, the new regime argument
indicates it was inadequate investment in production, not excess
investment in financial energy markets, that was primarily behind the
massive price increases of 2000 -- 2006.

In the oil market, many focus on the fact that spare crude oil
production capacity has diminished, and there have been additional
concerns over supply adequacy caused by the increasingly prominent "peak
oil" thesis.[^8] Such long-term concerns can explain why market
participants have bid up the price at the back of the forward curve.
Sellers at the back end of the curve may believe the peak oil argument
is overblown, and that in any event marginal cost does not set the crude
oil price.

A third and more nuanced view -- in some variations related to Peak Oil
-- argues that the world has exhausted most of the oil that is available
at finding costs of less than \$10/barrel. This leads to a traditional,
increasing-marginal-cost explanation for higher oil prices. The chart
above presents the relationship between production cost, oil already
produced, and the marginal costs of alternatives to "cheap oil" as seen
by the International Energy Agency (IEA). The IEA supply curve indicates
that there are 5 *trillion* barrels of oil available at "economic
prices" of less than \$70/bbl (in 2004 dollars). If this is correct, and
the oil extraction business still responds to economic opportunities,
then the market prices of \$70/barrel reached in 2006 were
unsustainable, and constituted a classic commodity "bubble." The
financial energy markets, by providing such convenient vehicles for the
financial expression of views about oil scarcity, will have contributed
to the bubble.

![](media/image2.wmf){width="6.506944444444445in"
height="3.310416666666667in"}The IEA supply curve is a useful tool for
pointing out that the quantity of "OPEC ME" (OPEC Middle East) available
oil is curtailed by instability (as with Iraqi oil), failure to maintain
fields properly (as some believe is the case with Iranian reserves), and
deliberate under-production of available reserves by governments who
have decided that their nations' discount rates are very, very low. The
conundrum is that there is still a great deal of oil in the "OPEC ME"
category, available for exploitation at less than \$15/bbl, but there
are political constraints on its expeditious production. Those who
invest in more expensive oil are essentially taking a political gamble
that this oil will continue to be held off the market, making it
economical to invest in the production of more expensive conventional
and unconventional oils. In essence, they are speculating on the
assumption that the sub-\$15 barrels are no longer on the margin.

For investors, these arguments are of more than academic interest. Any
investment in high-cost conventional oil resources runs the risk that
the constraints on producing more of the sub-\$15 oil pool will be
relieved. Such relief could come from resolution of political conflicts
in the Middle East, thus releasing the pent-up Iraqi reserves, or from a
decision by Saudi Arabia to double its production.

In the absence of liquid financial energy markets, market participants
could express their views on these issues in only two "physical"
markets: the spot market, and the markets for oil in the ground.[^9] The
existence of financial energy markets provides a forum for the
expression of views in the arena between spot markets and oil in the
ground. Financial energy markets are available for the "prompt month"
(which should converge with the spot price), and then for months and
years into the future.

**[Concerns About the Impact of Financial Markets]{.underline}**

As prices rose from the customary trading range of \$10 to \$30 in the
1990s, to \$30 - \$100 beginning in 2002, the debate about the role of
financial markets became more heated, and attracted notable
participants, especially in the United States. In the summer of 2006, a
number of investigative committees of the U.S. Senate conducted hearings
on the issue, and one committee---the Permanent Subcommittee on
Investigations -- issued an influential report titled *The Role of
Market Speculation In Rising Oil and Gas Prices: A Need To Put the Cop
Back On the Beat.*

Former Federal Reserve Chairman Alan Greenspan explained to the Senate
Foreign Affairs Committee that there had recently been "a major upsurge
in over-the-counter trading of oil futures and other commodity
derivatives." Greenspan observed that "increasing numbers of hedge funds
and other institutional investors began bidding for oil \[and\]
accumulated it in substantial net long positions in crude oil futures,
largely in the over-the-counter market," and that these activities
affected oil prices: "These net long futures contracts, in effect,
constituted a bet that oil prices would rise... With the demand from the
investment community, *oil prices have moved up sooner than they would
have otherwise*. "[^10]

Mr. Greenspan also suggested that these price increases have stimulated
additional oil production, a large increase in oil inventories, and a
partial scale-back of consumption.[^11]

The U.S. Senate sub-Committee investigating oil prices in 2006 concluded
that

> "\[T\]here is substantial evidence that the large amount of
> speculation in the current market has significantly increased prices.
> Several analysts have estimated that speculative purchases of oil
> futures have added as much as \$20-\$25 per barrel to the current
> price of crude oil, thereby pushing up the price of oil from \$50 to
> approximately \$70 per barrel."[^12]

These opinions about the effect of speculation in financial energy
markets on oil prices appears to be based on a fairly straightforward
proposition: the large and sudden increase in the market position of any
subset of oil market participants will tend to move prices up if the
increase is in demand (expressed in financial energy markets as an
increase in long positions) and down if the increase is in supply
(expressed in financial markets as an increase in short positions). Even
though there is "a long for every short and a short for every long," any
large market participant can create sudden surges in supply or demand
(or sudden shifts in the demand or supply curves of the oil market).

For example, assume that on a given day an extremely large speculator
decides to go short**.** His brokers will then attempt to purchase 5,000
short contracts. All other things being equal, the effect of such a
large increase in the number of shorts demanded is to drive down the
price. The order for 5,000 short contracts amounts to a search for 5,000
long contracts, and in the open outcry process on the floor of the
NYMEX, the bid price will fall until the necessary number of longs are
attracted to take the offsetting positions for the 5,000 shorts.

The nature of any market -- financial or physical -- is that such a
process can feed on itself. If the intra-day price decline forces prices
below a technical support level, those who trade on such signals will be
attracted into the market, creating further downward price pressure.
Market participants can wonder what led to the price decline if nothing
of consequence was indicated in the physical supply-demand equation.

For many economists, such effects can be expected only if two conditions
exist: a large market participant is exercising market power, or subsets
of market participants are (deliberately or coincidentally) acting
together, or "herding" and the effect of their acting together is to
exercise market power.[^13]

**[Trading Entities and Their Trading Styles]{.underline}**

While hedge funds have attracted much of the attention of the press,
they are only a subset of all of the institutions that trade oil. Even
within their own cohort, hedge funds are an extremely variegated herd,
each with its own directives and trading strategies. The effects of the
oil trades of these participants are unlikely to flow in the same
direction.

Quite apart from hedge funds, commercial and investment banks make a
variety of offerings to investors that, ultimately, result in a
financial institution placing substantial hedge positions in the market.
For example, some banks offer structured notes indexed on oil with fixed
or guaranteed returns. The buyers of these notes are not oil market
participants, and purchase them largely for portfolio diversification.

Of greatest recent interest is the role of yet a third type of trading
entity in the energy sector \-- index or passive investors, who are
looking for portfolio insurance via commodity returns, and are prepared
to pay for the portfolio benefit. These funds may buy an index, like the
Goldman Sachs Commodity Index, which guarantees them the return of the
index. The seller of that index, usually a bank, hedges the index
exposure in the energy market by buying the futures that correspond to
the index. Some participate as buyers in a Roll Index, which could
entail, for example, buying the second month futures. When the prompt
month expires, the Roll Index participant holds the prompt, which they
sell to buy the second month futures contract again.[^14] Others take
the opposite view, and are roll sellers.

There is a substantial amount of anecdotal evidence that much of the
increase in open interest in commodity contracts was precipitated by a
decision of pension funds to increase their exposure to commodities. For
example, the *Wall Street Journal* reported on September 9, 2004 that
"at the start of the decade, Europe\'s two biggest pension funds had no
commodity investments. Stichting Pensioenfonds ABP, the €157 billion
(\$190 billion) fund for Dutch government and educational-system
employees, got into commodities in 2001 and has reached its target of
having 2.5% of its assets, or close to \$5 billion, in commodities. A
year earlier, the €55.7 billion PGGM fund for Holland\'s
health-and-welfare sector waded into commodities. It now has 4.3%, or
roughly \$3 billion, invested in them."

The interest in commodity exposure was not restricted to pension funds:
"Harvard University has been investing in commodities for more than a
decade, and its internal benchmark calls for allocating 13% of the
university\'s \$19.3 billion endowment to oil, gas and other
commodities. That\'s just two percentage points less than the weighting
assigned to U.S. stocks and two points more than the allocation to U.S.
bonds. Unlike many pension funds that passively follow a benchmark,
Harvard actively manages its commodity investments, and 10 percentage
points of the 13 allocated to commodities are invested in timber."[^15]

Within the financial community, however, it would be quite unusual for
the trading activities of each of these entities to have effects in the
same direction. To the contrary, one would expect that the effects of
these traders on markets would *usually* be quite dispersed as each
trader follows his own objectives. Anecdotal accounts refer to a variety
of trading styles:

-   *Mean reversion* traders believe that commodity prices tend to
    return to long-term averages, and thus their trades would tend to
    help market prices remain within given channels. *Chartists or
    technical traders* could accentuate trends when their systems
    indicate they should endure, and bring them to an end to when their
    systems indicate they should not. Notions of powerful "support" and
    resistance levels have long held sway over this community, and there
    is an array of software support available to those who trade in this
    manner.

-   *Macro traders* believe in linkages between energy and other
    markets. These include incorporating views on macro-economic growth
    into energy trades (the "China syndrome"), or specific statistical
    relationships (dollar -- yen -- oil pricing dynamics).

-   *Fundamental commodity traders* who believe they have superior
    information about energy supply and demand.

-   *Passive investors* who only own energy contracts for their
    portfolio effects. In recent years, pension funds holding hundreds
    of billions of dollars began allocating a few percent to energy
    commodities. This so-called "index investing" is a trickle for this
    segment of the economy, however, is a torrent of investment for the
    energy sector. Banks make various offerings to their private banking
    investors that, ultimately, may result in a financial institution
    placing substantial hedge positions in the market (whether these are
    shown as hedges in CFTC reports is unclear). For example, some banks
    offer structured notes indexed on oil with fixed or guaranteed
    returns. The buyers of these notes are not oil market participants,
    and purchase them largely for portfolio diversification.

-   *Roll index* investors looking for portfolio insurance via commodity
    returns are prepared to pay for the portfolio benefit. They may buy
    an index, like the Goldman Sachs Commodity Index, which guarantees
    them the return of the index. The seller of that index, usually a
    bank, hedges the index exposure in the energy market by buying the
    futures that correspond to the index. Some participate as buyers in
    the Roll Index, which could entail, for example, buying the second
    month futures. When the prompt month expires, the Roll Index
    participant holds the prompt, which they sell to buy the second
    month futures contract again. Others take the opposite view, and are
    roll sellers.

**[Analysis of The Effects of Financial Markets on Oil and Gas
Prices]{.underline}**

In the past decade, various studies of the behavior of financial market
participants have been conducted, and they have shed a considerable
amount of light on what is, and is not, happening. In this endeavor, the
primary source of data on financial traders' behavior is the "Commitment
of Traders" report of the U.S. Commodities Futures Trading Commission
(CFTC).

The CFTC divides market participants into "commercial" and
"non-commercial" classes:

> "A trading entity generally gets classified as a \"commercial\" by
> filing a statement with the Commission (on CFTC Form 40) that it is
> commercially \"\...engaged in business activities hedged by the use of
> the futures or option markets.\" In order to ensure that traders are
> classified with accuracy and consistency, the Commission staff may
> exercise judgment in re-classifying a trader if it has additional
> information about the trader's use of the markets.
>
> A trader may be classified as a commercial in some commodities and as
> a non-commercial in other commodities. A single trading entity cannot
> be classified as both a commercial and non-commercial in the same
> commodity. Nonetheless, a multi-functional organization that has more
> than one trading entity may have each trading entity classified
> separately in a commodity. For example, a financial organization
> trading in financial futures may have a banking entity whose positions
> are classified as commercial and have a separate money-management
> entity whose positions are classified as non-commercial.
>
> The long and short open interest shown as \"Nonreportable Positions\"
> are derived by subtracting total long and short \"Reportable
> Positions\" from the total open interest. Accordingly, for
> \"Nonreportable Positions,\" the number of traders involved and the
> commercial/non-commercial classification of each trader are unknown."

Finally, as noted earlier in this report, the CFTC provides an
additional level of detail on the behavior of non-commercials with its
reports on spreading activities:

> "For the futures-only report, spreading measures the extent to which
> each non-commercial trader holds equal long and short futures
> positions. For the options-and-futures-combined report, spreading
> measures the extent to which each non-commercial trader holds equal
> combined-long and combined-short positions." [^16]

Published COT data, while interesting and useful, however, are
incomplete and sometimes unclear. The most important omission is
over-the-counter market transactions. As noted earlier, over the past
several years, more and more forward transactions (both energy and
non-energy) have moved from organized exchanges like NYMEX to platforms
for bilateral and OTC trades, particularly the Intercontinental
Exchange. Thus, it is appropriate to consider the volume and open
interest data of the NYMEX the tip of energy trading iceberg.

In addition, within the confines of the COT data, it is unclear how
bright the line is between the "non-commercial" (a.k.a., speculators)
and "commercials" (a.k.a., hedgers). Analysts disagree about the best
way to use the data in the analysis of the influence of specific trading
cohorts over prices. Some argue in favor of using total (commercial,
non-commercial, and non-reporting) open interest, others total
non-commercial open interest, and yet others total net non-commercial
open interest.

Even though the COT data are incomplete and unclear, they are all the
data practitioners have. Thus, given that is information, the analytical
community has naturally tried to find relationships between changes in
trading positions and prices. After some ten years during which a number
of studies -- with varying degrees of rigor and focus -- have been
conducted, it is fair to say that -- Robert Mabro's previous assertion
notwithstanding \-- there is no consensus on whether financial markets
exert a *sui generis* effect on the price of oil and gas.

Generally, formal statistical results on the issue are deemed
unimpressive, which is to be expected given the dispersion of trading
strategies already reviewed. A 2006 study conducted by the U.S.
Commodities Futures Trading Commission concludes that,

> "on average, \[managed money trader\] MMT participants do not change
> their positions as frequently as other participants, primarily those
> who are hedgers. We find that there is a significant correlation
> (negative) between MMT positions and other participant's positions
> (including the largest hedgers), and results suggest that it is the
> MMT traders who are providing liquidity to the large hedgers and not
> the other way around. We find that most of the MMT position changes in
> the very short run are triggered by hedging participants changing
> their positions. That is, the price changes that prompt large hedgers
> to alter their positions in the very short run eventually ripple
> through to MMT participants who will change their positions in
> response."[^17].

Robert Weiner, in a study published in 2006, obtained access to detailed
COT data and conducted an analysis focusing on market power and herding.
The study concludes that "the evidence indicates that speculators as a
group did not herd during the time period for which data are available
(mid-1990s). There is evidence, however, that some subgroups of
speculators do tend to act in parallel ('flock'), notably commodity pool
operators. .. Even among subgroups that flock, the extent of parallel
trading is modest."[^18]

Another study was conducted by analysts from the International Monetary
Fund, which compared the behavior of oil spot prices and the
non-commercial positions in three commodities: crude oil, copper, and
cotton. The IMF study -- acknowledging that COT data had numerous
characteristics that hindered definitive analysis --concluded that
"\[crude oil, copper, and cotton\] prices appear less volatile than
speculative positions across commodities, with no discernible common
trend between prices and speculation. For example, in the crude oil
market there has been no persistent pickup in net long noncommercial
positions in recent years when oil prices have had a strong upward
trend. More strikingly, in the copper market, net positions have
actually fallen steadily over the past year, during which prices have
reached record highs, suggesting that contrary to common perceptions,
speculation may not have played a major role in the recent price
run-up."

The IMF study left open, however, the possibility that -- for shorter
periods of time -- there may be a stronger link between the positions of
non-commercials and prices: "while the series do not appear to be
correlated over the long run, for most commodities some correlation
appears to be present over subperiods, as peaks and turning points seem
to occur around the same time across the two series."[^19]

**[Some Illustrative Analyses of Financial Market Effects on Oil
Prices]{.underline}**

If it is unsurprising that simple correlation studies would fail to find
a long-term, systematic relationship between oil prices and the changes
in trading positions, there remains an apparently large community who
appear less impressed with an absence of statistical proof and more
impressed with practitioner statements -- such as those by Mr. Greenspan
and Mr. Mabro -- that appear to take for granted that financial markets
have a *sui generis* impact on oil prices.

At the most general level, regulators appear to take for granted that
"the funds" can affect the markets in which they participate. For
example, the U.S. Federal Reserve Board's Patrick Parkinson testified
before Congress that "Although the role of hedge funds in the capital
markets cannot be precisely quantified, the growing importance of that
role is clear. Total assets under management are usually reported to
exceed \$1 trillion. Furthermore, hedge funds can leverage those assets
through borrowing money and through their use of derivatives, short
positions, and structured securities. Their *market impact* is further
magnified by the extremely active trading of some hedge funds. The
trading volumes of these funds reportedly account for significant shares
of total trading volumes in some segments of fixed income, equity, and
derivatives markets.[^20]"

The June 2006 report from the Senate Permanent Subcommittee On
Investigations titled "The Role Of Market Speculation In Rising Oil And
Gas Prices: A Need To Put The Cop Back On The Beat" cited (in addition
to Mr. Greenspan), Citigroup, Goldman Sachs, and eminent energy
economist Philip K. Verleger, Jr. as being in the camp of those who
believe there is a relationship between financial market activity
(specifically, speculation, which is a narrower form of participation
than we are reviewing in this paper), and oil prices.[^21]

Clearly, given the different ways financial institutions participate in
financial oil markets, simple regressions of the market positions of any
particular cohort of market participants (including "non-commercials")
with price changes are unlikely to be revealing. The trading community
is much too variegated to have a simple, measurable effect. The fact is
that financial markets offer a variety of participants, each with its
own directives and trading strategies. Therefore, the effects of the oil
trades of these participants would rarely flow in the same direction. An
analysis of the long-term relationships between any particular group of
traders and the price of oil or gas, therefore, is unlikely to provide
much insight. [^22]

More fundamentally, it would be naïve to expect any sustained causation
between trading strategies and prices. Some trading strategies are based
on the belief that markets eventually maintain fundamental
relationships, and trades reflecting that belief can act as stabilizers
to the market, and slow a market's adjustment to new developments.
Chartists and trend traders, in contrast, can push the market quickly to
new levels and can exaggerate price moves. Macro funds provide a linkage
between commodity markets and other global investment markets.
Fundamental commodity funds may actually enable futures prices to
reflect the current expected future outcome, where current publicly
available information is inadequate or inadequately distributed.

There are, nevertheless, several areas where causation should not be
dismissed, all of them consistent with normal economic analysis:

1.  *Perfect storm* episodes: there are likely to be periods of time
    when the condition of the physical energy market and trading
    strategies of financial market participants are in such good
    alignment as to produce "herding" and "bubbles" or their opposite,
    crashes.

2.  *Variations on the market power* *syndrome:* It is possible that the
    positions of some market participants -- index funds as one example
    -- are so large as to constitute witting or unwitting market power.
    A large-scale infusion or retreat from any of the various positions
    very large index funds might have price effects. The contract
    volumes involved in such shifts may \-- in the scale of oil trading
    \-- be quite large, but in the scale of money under management by
    these funds, be quite small. The index funds may be the "elephants
    in the bathtub" -- especially in the long-dated markets.

![](media/image3.png){width="3.75in"
height="2.6763779527559057in"}![](media/image4.png){width="3.75in"
height="2.6763779527559057in"}![](media/image5.wmf){width="3.75in"
height="2.4043307086614174in"}Analysts have traced developments in
[total]{.underline} open interest in the WTI futures contract and the
price of prompt WTI. At a simple level, the correlation of these two
data sets has an R^2^ of 0.6 over the entire 1983 to 2006 period. In
keeping with a theory that the financial markets occasionally have
little impact on oil prices, there are periods (e.g., from 1996 to
mid-1999) when there was a very low correlation between them, and other
periods (e.g., from January 2001 to October 2006) when the correlation
was quite strong (an R^2^ of 0.8). These results invite a theory that
(1) index funds may be biased to the long side of the WTI contract,
perhaps because a long position in oil is used as a hedge against
inflation, and (2) they tend to follow trends. That theory holds up
quite well during the bull market for oil and other commodities that was
evident from 2001 to 2006.

The analysis of the [net]{.underline} positions of non-commercials
reveals periods of greater and lesser correlation, *as is to be
expected* given the dispersion of trading programs. The charts nearby
define a net position as long minus short open interest, and break the
1986-2006 period into three parts:

1.  1986 to 1992, when (except for the period marking the Iraqi invasion
    of Kuwait and before the onset of the Gulf War that expelled Iraq
    from Kuwait) WTI was trading in a range of \$10 to \$20/bbl. During
    this period, non-commercials were usually [net short]{.underline}
    and their swings from net long to net short appeared to coincide
    with decreases in the price of WTI.

2.  1992 to 2001: Non-commercials' positions oscillated between net long
    and net short in long trading cycles that tracked a WTI trading
    pattern between \$14 and \$26/bbl. At the end of that cycle, during
    the period marked by the Asian financial crisis, WTI prices sank to
    nearly \$10/bbl.

3.  2002 to 2007: a great and extended bull market, beginning from the
    depths of the Asian economic crisis, brought oil prices from \$20 to
    \$100. The non-commercials were usually on the long side of this
    trend. Open interest in the WTI contract rose rapidly during this
    period. Anecdotal evidence indicated new money from index and
    pension funds, pulled into oil markets by financial advisors urging
    their clients to increase their long exposure to commodities. This
    inflow would have helped to accelerate the oil price increase. Their
    high rates of return in 2004 and 2005 fed the impulse of other funds
    to get on the long side of oil. This trend continued into 2006
    and 2007. When it appeared towards the end of the year that the long
    bull market was over, funds rapidly bailed out of their positions,
    accentuating the downward pressure on prices which pushed prices
    back towards \$80/bbl towards the end of the year.[^23]

In these charts, the visual correlation between the [net]{.underline}
positions of the non-commercials and the price of WTI
[occasionally]{.underline} supports a conjecture that sudden changes in
the preferences of speculators may be among the causes of changes in the
price of oil.

A third approach to the study of the effects of financial energy markets
is to evaluate the relationship between the spread positions in the
NYMEX WTI contract and the price of WTI. The volume of spread trades has
increased enormously, and now outweighs the number of outright trades.
It is interesting to observe that there is a high positive correlation
(R^2^ of 0.86) between the amount of open interest in spread trades
(futures and options) and the price of WTI. Without knowing more about
the structure of the time spreads (a study that goes beyond the scope of
this report but which might yield some interesting results), it is
difficult to know exactly what the implication of this correlation is.

Having reviewed these issues, it is important to note that neither hedge
funds nor any other cohort of the speculator community are infallible.
There are limits to the ability of speculators not only to anticipate
but also to influence markets. While WTI is an excellent vessel for
holding traders' fears and hopes, the very complex dynamics of global
crude oil physical demand and supply exerts itself inexorably, albeit
opaquely (because of the highly imperfect state of world crude supply
and demand data).

There is an inclination, especially among traders, to assume "the
financial market is always right." The record clearly shows, however,
that speculators are far from infallible. As the example of Amaranth
(and its predecessors like Long Term Capital and Metallgesellschaft),
they often make mistakes. Their mistakes are visible both in outright
and in spread trades. [^24]

ESAI has maintained a proprietary database of crude oil supply and
demand (aka, refinery intake of crude oil) since 1984. It has developed
several metrics of the condition of global crude oil supply and demand.
One of those metrics, ESAI's \"Crude Balance,\" which measures the
surplus or deficit of crude supply versus demand, indicated for the
winter and spring of 2003 that, while the speculators were aggressively
short-selling WTI, the global fundamentals were quite tight. At the same
time, there were plenty of political issues (tensions in the Middle
East, deteriorating supply conditions in some exporting countries) that
were keeping market participants -- with the exception of speculators in
the WTI market -- extremely apprehensive. Thus, the spring of 2003
represents a case where "the non-commercials" had a bearish view of the
market, but "the fundamentals" drove the market the other direction.

**[REGULATORY IMPLICATIONS]{.underline}**

Our ongoing evaluations of the ebb and flow of demand and supply in
financial energy markets has caused us to focus particularly strongly on
the activities of professional speculators. Speculators, such as
commodity funds, move in and out of the oil markets for reasons that may
have nothing to do with oil (e.g., because a trading program has noted
an historic propensity for oil to move one way when pork bellies move
another). It is possible that a sudden decline in demand for paper
barrels -- occurring because one or more large players suddenly decides
to abandon oil as a financial instrument -- causes a decline in paper
oil prices that quickly reduces the value of physical barrels.[^25]

Given this review, it appears to me that -- largely because of data
quality issues \-- the effects of financial markets on energy prices is
(to return to Gregory Treverton's useful distinction) a mystery rather
than a puzzle. As such, it is a problem that more amenable to an
"intelligence" approach to analysis (as in the Central Intelligence
Agency), than to a purely statistical approach. Because of the global
complexities of both the physical and financial energy markets, it is
unsurprising that purely statistical analysis of the relationship
between specific data sets (such as the open interest of
non-commercials) and energy prices.

How can energy market participants anticipate large and sudden shifts in
the financial energy markets? Not easily. At first glance, COT futures
data tell a simple story: there is always a short for every long. As the
volume and open interest change, then the long and short positions taken
by commercials, non-commercials, and small traders change from day to
day and week to week. As with physical market data, the COT data are
imperfect at best. They are simply clues in a complex detective story.
Why did non-commercials increase their short positions? Why did
commercials and/or small traders do the opposite?

We agree with Mr. Greenspan and others who believe organized speculators
now affect the value of many of the assets involved in world trade. No
one would object to this if only soft drink prices were affected by
their activities. But when they drive up the price of the fuels that
keep people from freezing, of the equities that hold the store of wealth
of savers, and of the currencies that determine a nation's stature in
world affairs, it is no wonder that we hear cries of alarm from
government officials.

In the United States, calls for reform of the rules governing
speculation tend to focus on two issues: *greater disclosure* and
*larger margin requirements*. On the disclosure front, the U.S.
Commodities Futures Trading Commission, during the tenure of the Bush
Administration, has been disinclined to apply futures-style disclosure
requirements on the burgeoning over-the-counter market.

The implosion of Amaranth, however, has rekindled calls for greater
disclosure.[^26] The most prominent specific measure that has surfaced
in Washington, DC is the "Oil and Gas Traders Oversight Act of
2006."[^27] The act -- only two pages in length -- simply requires that
"The \[CFTC\] Commission shall prescribe rules requiring such regular or
continuous reporting of positions in a *reportable contract* in
accordance with such requirements regarding size limits for reportable
positions and the form, timing, and manner of filing such reports under
this paragraph, as the Commission shall determine." A *reportable
contract* is defined as "a contract, agreement, or transaction involving
an energy commodity, executed on an electronic trading facility, or a
contract, agreement, or transaction for future delivery involving an
energy commodity for which the underlying energy commodity has a
physical delivery point within the United States and that is executed
through a domestic terminal."[^28]

Critics of this legislation contend that such disclosure will simply
cause the trading activity to migrate away from the United States. This
seems disingenuous: one of the reasons these markets use American
futures contracts as benchmarks is because they have faith in the
underlying integrity of the American financial system, in comparison
with the alternatives. There seems to us to be no compelling reason
[not]{.underline} to require such reporting, at least for US-centered
trade, and for European and Asian regulators to implement similar
reporting programs.

On the issue of *margin requirements,* we believe the influence of
speculators cannot be eliminated without throwing the baby -- the
efficacy of market forces -- out with the bath water. Government
officials and most well-informed market participants are aware that
markets cannot thrive without the depth and liquidity speculators add to
markets, without speculators to embrace the risk that others shun, and
without speculators' restless pursuit of new financial instruments that
-- in spite of all of the concern about derivatives -- still promote
financial flexibility and creativity. In short, speculators make markets
work better, albeit at a cost.

The cost, as both the academic literature and practitioners recognize,
is greater volatility. Markets with higher concentration of professional
speculators tend to exhibit greater price volatility. In the short run,
say, over a couple of months or at most a few years, this volatility is
inconvenient and even painful to those on the wrong side of the price
cycle. Thus, the relatively high oil prices of 2002 to 2006 were painful
for consumers, the low price of 1998 for producers.

![](media/image6.wmf){width="6.573622047244094in"
height="3.9188976377952756in"}In the long run, however, we find that
commodity prices and exchange rates in freely traded markets tend to
return to an economically rational level. Thus, oil prices -- thought in
the late 1970s and early 1980s to be on a constantly rising path -- in
fact showed a remarkable propensity to return to a very long-term
average level of around \$25 per barrel, as shown in the chart below,
which depicts oil prices since 1946 in inflation-adjusted terms. After
2000, the price ratcheted up to levels not seen (in real terms) since
the 1980 oil crisis. It appears from our analysis that the increase in
demand for paper oil barrels over the past several years helped to drive
those prices up. But if we have learned one thing about commodity
speculators over the years, it is that they are quite willing to trade a
commodity down as well as up. Thus, if the fundamentals of oil -- the
supply curve, the levels of demand, and inventories -- conspire to drive
short-term prices back down to that long-term \$25/barrel level,
speculators will ultimately help to get it there.

If the desire to regulate speculators further cannot be restricted to
just additional disclosures, the most direct and effective measure would
be to reduce their leverage by *increasing the margin requirements in
exchange traded markets*. Funds and exchanges, of course, will resist
this suggestion because it would adversely affect the liquidity of their
markets. And so it would, but if this segment of the financial community
does not regulate itself more effectively, chances increase that
governments will ultimately regulate them, perhaps out of existence.

It behooves the defenders of the new *status quo* -- market economists,
investment bankers, and Fund managers -- to do everything they can to
shed maximum light on these markets. There is still an astonishing lack
of disclosure of some of the information that helps make market forces
explicable. Until January 2007, for example, only the American futures
markets were required to collect commitment of traders data. In the
future, it is likely that OTC markets will have to do so[^29]. Such
vital market information as petroleum inventory levels are still not
published in a timely basis by the leading oil producing countries.
Saudi Arabia, the world's largest oil producer, fails to publish any
timely details of its operations[^30].

Thriving markets have an insatiable appetite for information. It is up
to governments -- the same ones that have surrendered some of their
controls to the market -- to ensure that adequate flows of information
exist to feed the markets to which they have entrusted their fates.
Governments must insist that those who sell critical commodities and
associated financial services -- whether it be Saudi Arabia or ICE or
NYMEX -- disclose enough information to ensure that known abuses (like
insider trading), and preventable problems (like development of market
power by an aberrant single hedge fund or herd of funds) do not fatally
undermine the efficacy of the markets. At the end of the day, markets
exist because governments allow them to. Support for oil, gas, and
electricity markets is ebbing, inside and outside the United States, and
advocates of markets as the best way to organize energy activities must
do all they can to shore up that support.

[^1]: We use *sui generis* here in its legal context: as a unique,
    distinct, or one-of-a-kind effect.

[^2]: ESAI first called attention to the apparently growing influence of
    financial markets in a memo we sent to our clients in 1994, which
    was subsequently published as an article in Energy Risk (November
    1994)?. Subsequent elaborations on our argument have appeared in
    memos to our clients as well as in later issues of *Energy Risk* and
    in *Managed Derivatives* (August 1996).

[^3]: Gregory Treverton, "Intelligence -- A Funhouse of Reflections,"
    Commentary on the website of the Rand Corporation;
    <http://www.rand.org/commentary/0216SFC.html> (January 17, 2007.

[^4]: Edward N. Krapels, "Hunters or Hunted?" *Managed Derivatives*, May
    1996, pp. 14-15.

[^5]: For a valuable summary, see *International Capital Markets:
    Developments, Prospects, and Key Policy Issues*, International
    Monetary Fund, (a Staff Team led by Charles Adams, Donald J.
    Mathieson, Garry Schinasi, and Bankim Chadha). Available on the
    World Wide Web at
    http://www.imf.org/external/pubs/ft/icm/icm98/index.htm

[^6]: "A Hedge Fund Falters, and Big Banks Agree to Ante Up \$3.5
    billion," *Wall Street Journal,* September 24, 1998, p. A-1.

[^7]: Robert Mabro, "Robert Mabro Questions the Suitability of the
    Current Oil Price Regime," *Oxford Energy Forum*, February 2007,
    page 15.

[^8]: Those who adhere to the Peak Oil argument have developed , *inter
    alia,* regular conferences and journals reviewing the topic. See the
    Peal Oik web site maintained by Colin Campbell at
    <http://www.peakoil.net/>.

[^9]: While derivative markets for oil have existed in some forms since
    the 19^th^ century, we will date the emergence of the modern
    petroleum derivatives markets with the successful launch of the
    forward markets in the North Sea in the late 1970s and the
    successful launch of the New York Mercantile Exchange crude oil
    contract in 1984. We treat investments in oil producing companies as
    a subset of the market for oil in the ground.

[^10]: Greenspan, Alan. *Statement before the Senate Committee on
    Foreign Relations, United States Senate*, June 7, 2006.

[^11]: Coleman, Norm (Chairman). Levin, Carl (Ranking Minority Member).
    *The Role of Market Speculation In Rising Oil and Gas Prices: A Need
    To Put the Cop Back On the Beat*, Staff Report, Permanent
    Subcommittee on Investigations, U.S. Senate, June 27, 2006, page 3.
    Hereafter, Coleman \[2006\].

[^12]: Coleman, Norm (Chairman). Levin, Carl (Ranking Minority Member).
    *The Role of Market Speculation In Rising Oil and Gas Prices: A Need
    To Put the Cop Back On the Beat*, Staff Report, Permanent
    Subcommittee on Investigations, U.S. Senate, June 27, 2006, page 2.

[^13]: For a detailed presentation on the difference between these two
    market characteristics as it applies to financial oil markets, see
    Robert J. Weiner, *Do Birds of a Feather Flock Together: Speculator
    Herding in the World Oil Market*, (Washington DC: Resources for the
    Future Discussion Paper, June 2006).

[^14]: See, for example, Dizard, John, "Speculators Profit from
    Commodity Investors," which notes that speculators "In the pits,
    physical or electronic, bet against the certainty that commodity
    index investors' positions are rolled in a mechanistic manner every
    month, in known patterns on particular days." *Financial Times,*
    Jan. 22, 2007.

[^15]: Michael R. Sesit, "Commodities Enter Investment Mainstream,"
    *Wall Street Journal,* September 9, 2004.

[^16]: See "Backgrounder" on CFTC web site
    http://www.cftc.gov/opa/backgrounder/opacot596.htm.

[^17]: See Michael S. Haigh, Jana Hranaiova and James A. Overdahl,
    Office of the Chief Economist, U.S. Commodity Futures Trading
    Commission, "Price Dynamics, Price Discovery and Large Futures
    Trader Interactions in the Energy Complex," April 28, 2005.

[^18]: Weiner, *Do Birds of a Feather Flock Together: Speculator Herding
    in the World Oil Market* (Washington, DC: Resources for the Future
    Discussion paper, June 2006).

[^19]: Sergei Antoshin and Hossein Samiei, ""Has Speculation Contributed
    to Higher Commodity Prices?" in Name of IMF Report, (Washington, DC:
    International Monertary Fund), pp. 15-18.

[^20]: Patrick Parkinson, Deputy Director, Federal Reserve Board,
    Division of Research and Statistics, "The Role of Hedge Funds in the
    Capital Markets," Testimony Before the Subcommittee on Securities
    and Investment, Committee on Banking, Housing, and Urban Affairs,
    U.S. Senate May 16, 2006. Emphasis added.

[^21]: Coleman \[2006\], p. 24-25.

[^22]: Economists call the tendency of traders to engage in similar
    strategies "herding." Robert Weiner has shown that there is little
    evidence of systematic and prolonged herding in the financial oil
    markets.

[^23]: It is important to note that the charts match *weekly* COT data
    with the price of prompt (first month) WTI. Correlation statistics
    are highly sensitive to the time period chosen, which is consistent
    with a theory that the influence of financial markets on oil prices
    waxes and wanes. The inter-relationship between events in the
    physical market and financial markets are extremely complex.

[^24]: Amaranth reportedly lost \$3 billion in a period of a few months
    by trading in the hyper-volatile U.S. natural gas market For an
    account, see "Betting the House and Losing Big," *The New York
    Times*, Sep, 23, 2006.

[^25]: Newspaper accounts in January 2007 indicated that "Wall Street
    commodity funds that have been investing heavily in energy futures
    are now loading up on agricultural commodities like corn and
    livestock futures." See "Wall Street is Betting on the Farm," *New
    York Times*, January 19, 2007.

[^26]: See in particular, Coleman and Levin, *The Role of Market
    Speculation In Rising Oil and Gas Prices: A Need To Put the Cop Back
    On the Beat*, Staff Report, Permanent Subcommittee on
    Investigations, U.S. Senate, June 27, 2006, page 3.

[^27]: The influential business journalist of the *New York Times,*
    Gretchen Morgenson, called for passage of the Act in a September 24,
    2006 article titled "Dangers of a World Without Rules."

[^28]: *Oil and Gas Traders Oversight Act of 2006*, available at
    http://thomas.loc.gov/cgi-bin/query/z?c109:S.2642:.

[^29]: On January 30, 2007, *Platts Energy Trader* reported that ICE was
    reporting large gas trading positions to the FTC on a daily basis.

[^30]: See, however, www.JODIDATA.org for information about the Joint
    Oil Data Initiative, which is headquartered in Saudi Arabia.