Speculation, Fundamentals, and Oil Prices
Released: 2011-09-02T00:00:00

Introduction

High oil prices affect nearly every household and business in the United States. Figure 1 illustrates that during the course of 2008, oil prices doubled to more than $145 per barrel and then rapidly fell by 80%. In early 2011, there was a run-up of about 20%, sending gasoline prices near 2008 highs. Few would rule out the possibility of similar price swings in the months to come. What explains oil price volatility?

There are two possible kinds of explanations. The first looks to the fundamentals of oil production and energy consumption. Rapid global economic growth has led to rising demand for oil, and supply could not keep up at previous oil prices. But oil supply and demand are inelastic to price changes, at least in the near-term, which some would argue means that relatively small shifts in supply or demand can be expected to trigger significant price movements.

Others consider price movements such as those of 2008 and early 2011 to be more extreme than warranted by the fundamentals of supply and demand. The second explanation for unstable commodity prices focuses on financial markets for derivatives contracts that are linked to the price of oil—futures, options, and swaps. Many market participants are pure financial speculators, who never deal in physical oil, but seek to profit from correctly forecasting price trends. Critics claim that speculators can drive oil prices above fundamental levels, resulting in a "speculative premium" that imposes unjustified costs on consumers.

Although the relationship between speculation and commodity prices has been studied extensively, there is no consensus among academics and regulators as to whether speculative trading causes episodes of unusual price volatility. This report provides background on the oil derivatives markets and the different types of firms that trade in those markets. It reviews the concepts of manipulation and excessive speculation. It includes a brief section describing the fundamental factors that affect oil prices. Although the basic question of fundamentals versus speculation remains unsettled, this report provides a context for evaluating the opposing claims.

Oil Markets, Prices, and Derivatives

The United States consumes about 19 million barrels of oil each day. Maintaining the supply of oil involves thousands of daily transactions, at prices varying by location, quality, quantity, and local supply and demand conditions. There is no single price at which spot (or physical) oil deals take place, but there are a number of benchmarks that buyers and sellers use as reference points. Private and government sources publish benchmark data about spot prices at various locations.

Another important oil price benchmark is the futures price. A futures contract is a form of oil "derivative"—it is a financial instrument that gains and loses value as the price of oil rises and falls. In effect, futures traders buy the price of oil (and make or lose money as the price changes) without necessarily delivering or taking possession of a single barrel of the physical commodity. Thousands of traders buy and sell oil futures contracts. Their purposes, strategies, and investment horizons vary, but the sum of their transactions determines the futures market price, which is publicly available to all market participants. Many spot trades take place at the futures price, or at the futures price adjusted by some factor. Headlines reporting a dramatic jump or fall in oil prices are likely to quote the futures price.

Both physical and derivative trades (futures, options, and swap contracts linked to the price of oil) contribute to setting the price. It is thus very difficult to disentangle the price impact of trades by producers and commercial users of oil from those of financial speculators who seek to profit by forecasting price trends. Does excessive speculation drive prices away from levels justified by supply and demand fundamentals, or do speculators provide liquidity and facilitate the price-setting mechanism? These questions remain controversial. The next sections of this report describe the mechanics of oil futures and the kinds of traders who participate in the market. Although oil swaps and options use different terminology, the economic substance is the same: they are bilateral contracts under which one party gains if the price moves in one direction, and the other party gains if the price moves in the opposite direction.

The Mechanics of a Futures Contract

An oil futures contract represents 1,000 barrels of oil, but neither party to the contract need ever possess the actual commodity. (Contracts may be settled by physical delivery, but in practice the vast majority are settled in cash.) When a futures contract is made today, one party (called the "long") agrees to buy oil at a future date from the other (the "short").

Contracts are available with different maturities, designated by various expiration months, but the size is always the same. (In crude oil, a contract expires every month and most trading is in the contract soonest to expire, called the "near" or "spot-month" contract.) The price at which the future purchase or sale is to take place is the current futures market price, which varies continuously during the trading day. Assuming the price of oil is $100 per barrel, the long trader is committed to buy at that price, and the short is obliged to sell.

Now suppose that tomorrow the price of oil goes to $105/barrel. The long trader now has the advantage: he is entitled to buy for $100 oil that is now worth $105. His profit is $5,000 (the $5 per barrel increase times the 1,000 barrels specified in the contract). The short has lost the identical amount: she is obliged to sell oil for less than the going price.

If, on the following day, the price goes to $110, the long gains another $5,000. The short, down a total of $10,000, may reconsider her investment strategy and decide to exit the market. She can do this at any time by entering into an offsetting, or opposite transaction. That is, she purchases a long contract with the same expiration date. Her obligation (on paper) is now to sell 1,000 barrels (according to the first contract) and to buy 1,000 barrels (the second contract) when both contracts expire simultaneously. Whatever price prevails at that time, the net effect of the two transactions will be zero. The short's position is said to be "evened out"—she is out of the market.

The short's decision to exit does not affect the long, who may prefer to ride the trend. This is because all contracts are assumed by the exchange's clearing house, which becomes the opposite party on each trade, and guarantees payment. The ability to enter and exit the market by offset, without having to make or take delivery of the physical commodity, permits trading strategies based on short-term price expectations. While some traders may keep a long or short position open for weeks or months, others buy and sell in fractions of seconds.

The exchange clearing house, which guarantees all trades, also controls traders' funds. Before entering into the trade described above, both long and short would have been required to deposit an initial margin payment of $6,750. (The figure, set by the exchange, was the CME Group's margin for speculators as of August 31, 2011.) All contracts are priced, or "marked-to-market," each day. The long trader above would have had his $10,000 gain credited to his margin account, whereas the short would have had to make additional "maintenance" margin payments to cover her losses. It is worth noting that her two-day $10,000 loss represents more than 100% of her original investment, that is, her initial margin deposit of $6,750: the risks of futures speculation are high. When traders exit the market, any funds remaining in their margin accounts are returned. (Other transaction costs, such as brokerage commissions and clearing fees, are not returnable.)

Options on futures are also available for many futures contracts. The holder of an option has the right (but not the obligation) to enter into a long or short futures contract over the life of the option. The option will only be exercised if price movements are favorable to the option buyer, that is, if the underlying futures contract would be profitable. The seller of the option receives a payment (called a premium) for granting this right. The seller profits if the option is not exercised by the buyer.

Swap contracts are traded in the over-the-counter market, rather than on organized exchanges (although the Dodd-Frank Act Reform and Consumer Protection Act, P.L. 111-203 , mandates that some swaps be traded on new "swap execution facilities," or SEFs). The terms of swap contracts are not uniform, as futures contracts are, but can be negotiated between the counterparties. Economically, however, swaps are equivalent to futures: one counterparty will gain if the price rises, the other if prices fall.

Who Trades in Oil Derivatives Markets

Derivatives traders can be classed as either hedgers, who use the market to avoid price risk, or speculators, who assume risk in search of profits. In futures markets the distinction is formal, and is important because hedgers pay lower margin rates and are not subject to limits on the size of their positions. Hedgers and speculators may be further broken down into subcategories, as follows.

Commercial Hedgers

Commercial hedgers are those involved in production, processing, transportation, or use of oil and petroleum products. Oil and gas exploration and production companies, refiners, industrial consumers, and retailers buy and sell oil and oil products to meet the physical needs of their businesses. In their physical trading, they buy and sell oil up and down the supply chain. For example, an upstream producer sells crude oil to a refinery, which sells jet fuel to an airline or gasoline to a retail station, which then sells it to motorists. Commercial participants can sign long-term sales agreements or may buy short-term contracts for near-term physical delivery of oil.

Derivatives allow commercial participants to manage their risks related to the oil business, or hedge against oil price risk. This is a form of insurance against market fluctuations. For instance, an airline's profits may suffer when jet fuel prices increase. To address this risk, the airline can purchase long derivatives contracts whose value rises when oil or jet fuel prices increase. If prices then do increase, the cost of higher-priced fuel is offset by the money gained on derivative contracts. Alternatively, an upstream oil company can obtain a short derivative to insure against lower future oil prices.

Swap Dealers

Swap dealers are entities that deal primarily in swaps for a commodity and use the futures markets to hedge risk associated with those swap transactions. For example, a pension fund wishing to include commodities in its investment portfolio might enter into a swap linked to a published commodity price index. If the index goes up, the dealer will owe the pension fund money. To hedge that risk, the swap dealer may take an equivalent (but opposite) position in the futures market. Then any payment due to the swap counterparty will be offset by earnings on the futures position.

A swap dealer's counterparties may be speculative traders, like pension funds or hedge funds, or producers and commercial users that are hedging risks of dealings in the physical commodity. (Many hedgers prefer swaps to futures because swaps can be customized to fit the exact quantities and time frames relevant to the hedger's business, whereas futures have uniform contract sizes and expiration dates.) Thus, swap dealer positions represent both hedging and speculation.

Money Managers

Money managers, a group of purely speculative traders, are professionally managed funds trading on behalf of clients. Money managers who invest in futures generally must register with the CFTC as commodity trading advisors (CTAs) or commodity pool operators (CPOs). The money manager class includes hedge funds, which invest on behalf of institutional investors (such as pension funds) and wealthy individuals.

Other Speculators

Other kinds of speculators include floor traders, or exchange members who trade for their own accounts, as well as a variety of firms and wealthy individuals. Small, public investors are able to trade futures, but the retail presence in futures is likely much lower than in the stock market.

The industry uses different terms for speculators with different time horizons. "Scalpers" take very short-term positions: minutes, seconds, or less. High-frequency trading, where the relevant time unit is the microsecond, is making inroads into derivatives trading, just as it has in the stock market. "Day traders" may hold contracts for longer intervals, but they liquidate their positions before the close of trading, to avoid exposure to overnight price risk. (As a global commodity, oil trades around the clock.) "Trend followers" may hold positions open for days, weeks, or longer, attempting to profit based on their expectations of long-term price trends.

Speculation and Hedging in Oil Futures Markets

There are no public data on how much oil futures trading is speculative, although the assumption is that speculators account for most short-term trading, which in turn accounts for most market turnover. The Commodity Futures Trading Commission (CFTC), however, publishes weekly Commitments of Traders (COT) reports, which present data on the size of positions held by several kinds of market participants. COT data, usually published late afternoon each Friday, reflect the open interest, or the number of contracts outstanding, as of close of trading on the previous Tuesday. Thus, comparing week-to-week COT figures shows whether classes of traders have increased or decreased the size of their long, short, or spread positions. The COT figures do not show how much trading has gone on during the week, or whether a position has been liquidated and then built back up, but simply offer a snapshot of positions at the market close on Tuesday.

Another significant limitation of COT data is that they do not cover swap contracts—another form of oil derivative contract not traded on exchanges. Thus, COT figures arguably cover only a subset of oil derivatives, all of which play a role in setting prices. The Dodd-Frank Act ( P.L. 111-203 ) gave the CFTC new regulatory authority over the swaps market. In the future, COT reports may reflect swap positions, but the data currently available cover only exchange-traded futures and options on futures.

Table 1 breaks down open interest in crude oil futures and options on futures as of July 19, 2011. The figures represent the sum of identical contracts traded on the New York Mercantile Exchange (or Nymex, part of CME Group located in New York) and ICE Futures Europe (based in London). Both contracts reference West Texas Intermediate (WTI) crude, also called "light, sweet" oil, as traded in Cushing, Oklahoma (a major pipeline junction).

The data in Table 1 prompt several observations about the market:

Speculators appear to hold most of the open interest in crude oil contracts. Producer/merchant hedgers account for only 19%; only part of swap dealers' 38% represents hedging interests; and non-reportable contracts (which may be either speculative or hedges) are less than 4%. The remainder is held by speculators. No class of trader has a clearly dominant market share, either long or short. Over half of all contracts are part of spread positions, involving simultaneous purchases of long and short contracts (with different expiration months). Spread positions are less risky and offer less potential reward than outright short or long positions. Managed money positions, which include hedge fund investments, account for a fairly small share of total open interest, but are heavily concentrated on the long side. This means that they profit when prices rise. Reportable positions—those with at least 350 contracts—account for more than 95% of all open interest. This suggests that small, retail investors play a minor role in oil futures markets. The number of reporting traders in each category is fairly small, compared (for example) with stock and bond markets, where many thousands of individuals and institutions have significant positions.

Price Impact of Speculation

In June 2011, the CFTC released a one-time report on large trader net position changes in Nymex crude oil futures, supplementing the COT reports. This data set covers the period between January 2009 and May 2011, and it shows (on a weekly basis) the daily average of net position changes for both long and short positions for each of the categories of traders shown in Table 1 . The figures show the amount by which long traders increased their long positions (net buys) and the amount by which short positions were increased (net sells).

Thus, for each week and for class of trader, the data show whether on average long positions (buys) exceeded short position increases (sells), or the reverse. Figure 2 presents (1) the net figure of buys and sells for managed money trading, which includes trades of hedge funds, commodity pool operators, and others; and (2) changes in the price of oil during the same period. Each point in the graph represents a single week's change in these two figures: the net average sales or purchases by money managers and the price change over the same week.

The horizontal and vertical axes divide Figure 2 into quadrants. Data points located in the upper right indicate weeks when money managers were net buyers and the price of oil rose. Points in the lower left indicate weeks when the price dropped and money mangers were net sellers. The other two quadrants indicate weeks when prices rose and money managers sold or when prices fell and they were net buyers: in other words, when their transactions and the price moved in opposite directions.

Figure 2 suggests that there is a correlation between money manger transactions and price movements. Weeks in which the price rose sharply tended to be when they bought heavily. The more prices fell, the more they tended to sell. Very few data points fell into the upper left quadrant, that is, money managers were rarely net buyers when prices were falling.

Indeed, the results of regression analysis, given in Appendix A , show that a strong and statistically significant correlation does exist between money manger transactions and price movements. (Please see Appendix A for details of the regression.)

Figure 3 shows the same data for the trades of commercial hedgers (the group called "producer/merchants" in Table 1 ). Here, there appears to be no correlation, or trend-line. Neither is there any apparent correlation between the trades of (1) swap dealers or (2) other speculators and price movements, as shown in Figure 4 and Figure 5 .

Figure 2 suggests that crude oil futures are not a textbook case of an efficient market, where prices incorporating all known information about the commodity move in a random walk. The group of speculators classified as money managers appears able either to anticipate price movements or to cause those price movements through their trades. This observation raises some interesting questions. Why should money managers be better forecasters of oil price movements than other speculators or commercial hedgers? Given that their long and short positions constitute a small share of the total market, why should money manager trades have a unique price impact? Most fundamentally, are money managers' trades determining prices or are they simply more adept than others in following trends or identifying information and news that will drive prices up or down?

Assuming that money managers have a unique impact on price, what is the mechanism by which their transactions—relatively small in terms of the total market—move prices? A possibility is that they affect intraday trading, which the available open interest data fail to capture. Short-term traders might observe and seek to copy the strategies of certain money managers who are regarded as especially capable of identifying new information that might be expected to move prices, or who simply have achieved superior returns in the past. If significant numbers of short-term speculators copy money manager trades, the impact of those trades on prices would be magnified. In effect, under this scenario, money managers may have market power beyond what the size of their positions would suggest. If managed money trades trigger a significant number of similar transactions by others, they become a kind of self-fulfilling prophecy. Such "herding behavior" among speculators, if it exists, would support arguments that the oil price at times includes a "speculative premium" above and beyond the price justified by the fundamentals.

On the other hand, it may be that money managers do trade on fundamental information and that they are especially skilled at identifying news that is going to move prices. If money managers are consistent in their ability to identify new and relevant information that will affect prices (and trade on that information before others do), one result would be the observed correlation. A potential objection to this explanation is that it implies that some financial speculators are better analysts of the oil market than actual producers and end-users of oil, who also trade in the futures market.

Money managers might also profit by following price trends. Rather than cause price changes, they may buy when they see prices are rising and sell when prices begin to fall. But why would money managers' trading patterns, and not those of other market participants, be correlated with price changes in this way?

Other market participants may have longer investment time horizons or be less sensitive to price changes. Hedgers, for example, are generally less affected by price changes, because whatever they may lose on their futures positions, they make back in the spot market (because, for example, the physical commodity they produce will have gone up in price). Similarly, swap dealer positions may reflect long-term commodity index investments by pension funds and other institutional investors who are seeking to allocate part of their portfolio to an asset class that is not correlated to other assets they hold, such as stocks and bonds. Because the object of such investment is portfolio diversification, such investors are less likely to buy or sell in reaction to short-term price movements.

Hedge funds, by contrast, are known for taking aggressive positions in search of high yields and for seeking to extract the maximum return from any price trend. A 2008 CFTC study referred to speculators "who take positions based on price expectations over a period of days, weeks, or months" as "trend followers." Trading with this time horizon would be captured by the weekly COT reports and the net position change data in Figures 2 through 5 , and may be more common with money managers than other traders in oil futures and options.

If money manager trades can be said to cause price movements (that is, if we assume that such trades cause price changes, rather than follow them), are they responsible for long-term price changes such as the run-up of prices in the first half of 2008? The data released by the CFTC do not support that conclusion. When weekly position changes are plotted against changes in price in the following week (instead of the same week, as in Figures 2 though 5 ), the correlation essentially disappears. In other words, managed money trades may cause prices to rise or fall in the week they are made, but they do not appear to trigger longer price trends.

The same is true over other time horizons. For example, Figure 6 shows changes in money manager positions and price changes four weeks later. The data suggest that there is no correlation between whether hedge funds and other money managers buy or sell this week and what happens to prices over the next month.

Derivatives Markets and Price Distortions

If derivatives speculators have mispriced oil during recent years, there are two ways this could have happened. The first is through deliberate manipulation of the price by a group of market participants. Knowing action to create artificial prices is a violation of the Commodity Exchange Act, and the regulators and exchanges have market surveillance programs to detect attempted manipulation. The second possibility is much harder to evaluate: do derivatives markets in their normal operation have the potential to distort prices? Section 4a(a) of the Commodity Exchange Act describes "excessive speculation" as "an undue and unnecessary burden on interstate commerce," but there is no statutory or regulatory definition of the term. There is an extensive literature on the relationship between speculation and commodity prices, but the question is not settled—the data are subject to conflicting interpretations.

Manipulation

The Commodity Exchange Act (CEA), the statute which governs the regulation of commodities and futures markets, makes it a felony to manipulate or attempt to manipulate the price of a commodity, including one for future delivery. Yet nowhere in the CEA does there exist a definition of the term "market manipulation." Instead, details of what it means to manipulate a futures or commodity market, in practice, have been fleshed out over the years through court decisions and regulatory actions by the CFTC.

Courts have determined that manipulation must include the following three elements:

at the time of the alleged manipulation, there was an "artificial" or "abnormal" price in the futures market; at the time such artificial price existed, the alleged manipulator had a dominant enough market position to permit the conclusion that he caused the artificial price; and the alleged manipulator intended to cause the artificial price.

Section 753 of the Dodd-Frank Act ( P.L. 111-203 ) provides the CFTC with additional anti-manipulation authority—providing false reports concerning market information that could affect prices is made a violation; anti-manipulation provisions are applied to the swap market; and criminal penalties are increased.

On April 21, 2011, President Obama announced that the Attorney General was assembling a team to root out any fraud and manipulation in the oil markets that might be contributing to higher U.S. gasoline prices. The team includes representatives from the CFTC, the Federal Trade Commission (FTC), the Federal Reserve Board, the Securities and Exchange Commission (SEC), as well as the Departments of Agriculture, Energy, Justice and Treasury.

In May 2011, the CFTC filed an enforcement action against three energy trading firms and two individuals, charging them with a series of manipulations between January and April 2008 (a period when prices were rising rapidly). To execute the scheme, the defendants are alleged to have created an artificial shortage of deliverable oil in Cushing, Oklahoma on several occasions when Nymex futures contracts were due to expire. They did so by buying large quantities of physical oil, causing market participants to revise downwards their estimates of the amount of oil available to settle maturing futures contracts. This perception of limited available supply drove up the price, allegedly earning the defendants profits from a long position in futures. Then, having taken profits from the long position, the traders liquidated their physical oil holdings very rapidly, depressing the price and allowing them to profit from a short position in futures.

According to the CFTC, the defendants lost about $15 million on the physical side of these transactions, but earned about $50 million on the derivatives side. Their futures positions were calendar spreads—a long contract for one month and a short contract for another month. The CFTC alleges that the profitability of the scheme depended on the defendants being able to manipulate the price differentials between the two contracts in the February/March and March/April 2008 spreads. According to the complaint, the spread between the February 2008 and March 2008 Nymex futures contracts widened from $0.24 on January 3, 2008, to $0.64 on January 22, 2008.

The CFTC complaint does not state how much or even whether the alleged scheme affected the price of oil itself—since the defendants were trading spreads, the absolute price level was less important to them than the difference between the various month contracts. According to the CFTC, the manipulation was put in motion on January 3, 2008, and ended on April 17, when the defendants received a request for documents from the CFTC. During that period, the price of oil rose from $99.17 to $114.80. Price increases accelerated after the manipulation ended: crude oil rose by $15 per barrel in May alone, and by another $12 in June.

Thus, although manipulations do occur in futures markets, this case (even if all allegations are proven) appears to involve short-term price dislocations that do not explain the price run-up in 2008, as it continued after the alleged manipulation ended.

Speculation

Theories of Speculation

There are two opposing theoretical views on speculation. The first is that it tends to stabilize prices and make the price-setting mechanism more efficient; the second is that at times speculation causes price trends that cannot be explained by fundamental economic factors.

The view of speculation as a stabilizing force is expressed as follows by Milton Friedman:

People who argue that speculation is generally destabilizing seldom realize that this is largely equivalent to saying that speculators lose money, since speculation can be destabilizing in general only if speculators on the average sell when the [commodity] is low in price and buy when it is high.

To make money, speculators must be able to buy low and sell high. By so doing, they smooth out the peaks and troughs of commodity prices. If they are unable to do this successfully, if their price forecasts are more often wrong than not, they will be driven out of the market by their losses.

This benevolent view of speculation is generally supported by empirical research into the effects of futures trading on cash market prices. Although the issue cannot be regarded as settled, numerous studies have found that the existence of a futures market either has no effect on or tends to reduce price volatility in the underlying commodity. For example, a recent Federal Reserve study compared price movements over the period 1991-2008 in industrial metals for which there is a futures market and metals for which no futures contract exists. The study found that prices for "traded and non-traded metals are positively correlated" and that "the intensity of speculative activity in the futures markets has no explanatory power for metal price growth rates.... Instead, commodity spot price changes are driven by world economy activity and financial investors are merely responding to these price changes."

In 2008, an Interagency Task Force formed by the CFTC studied price movements in crude oil, and reached a similar conclusion:

The Task Force's preliminary assessment is that current oil prices and the increase in oil prices between January 2003 and June 2008 are largely due to fundamental supply and demand factors. During this same period, activity on the crude oil futures market—as measured by the number of contracts outstanding, trading activity, and the number of traders—has increased significantly. While these increases broadly coincided with the run-up in crude oil prices, the Task Force's preliminary analysis to date does not support the proposition that speculative activity has systematically driven changes in oil prices.

More specifically, the report found that "changes in futures market participation by speculators have not systematically preceded price changes. On the contrary, most speculative traders typically alter their positions following price changes, suggesting that they are responding to new information—just as one would expect in an efficiently operating market."

From an opposing theoretical perspective, speculation is seen as a potential source of price instability. Describing the behavior of investors, J.M. Keynes distinguishes between enterprise, the activity of forecasting the long-term yield of assets, and speculation, the activity of forecasting the psychology of the market. As speculators, he wrote, " ... we devote our intelligences to anticipating what average opinion believes average opinion to be." In a market dominated by speculation of this type,

A conventional valuation which is established as the outcome of the mass psychology of a large number of ignorant individuals is liable to change violently as the result of a sudden fluctuation of opinion due to factors which do not really make much difference to the prospective yield.

More fundamentally, Keynes wrote that "when the capital development of a country becomes a by-product of the activities of a casino, the job is likely to be ill-done." The negative view of commodity speculators is that they may trade on information unrelated to the fundamentals of supply and demand and, in the process, generate prices that harm consumers and volatility that creates uncertainty throughout the economy.

If speculators bring new fundamental information to the market, their trading should not only be profitable but should make the price discovery mechanism more efficient. When prices appear to diverge from economic reality, when a price bubble forms, many conclude that speculators are distorting the price-setting mechanism. There are several explanations for how price bubbles expand—irrational exuberance, positive feedback, herding, and so on—but the process by which bubbles start and end remains little understood.

How is it possible to know at any given moment whether speculation is playing a stabilizing role or whether markets are behaving irrationally? Which model of speculation best describes reality? The Commodity Exchange Act states that speculation may distort prices when it becomes excessive. The term "excessive speculation" does not provide a precise tool for distinguishing between beneficial and harmful speculation—"excess" is in the eye of the beholder—but it does provide a framework for analyzing the impact of speculation on the oil market.

Excessive Speculation

Section 4a(a) of the Commodity Exchange Act declares that "[e]xcessive speculation in any commodity ... causing sudden or unreasonable fluctuations or unwarranted changes in the price of such commodity, is an undue and unnecessary burden on interstate commerce in such commodity." Unlike manipulation, however, excessive speculation is not a violation of law. The point at which speculation becomes excessive is left to the regulator to determine: there is no statutory definition or benchmark.

To many observers, phrases like "sudden or unreasonable fluctuations" and "unwarranted changes in the price" aptly describe the oil markets of 2008 and 2011. When oil prices are high and volatile, and there have been no dramatic supply shocks, many blame financial speculation.

Arguments that Oil Speculation has been Excessive

The case against oil speculators, or rather the case that oil speculation has become excessive, rests principally on two arguments. First, there is said to be too much speculative trading. While it is generally acknowledged that hedgers benefit from the presence of speculators willing to take on the risks that hedgers wish to avoid, the argument is made that financial traders have overwhelmed the market. Rather than simply provide liquidity to hedgers, speculators now account for the majority of contracts. As Table 1 shows, commercial hedging positions account for less than half of all crude oil contracts outstanding. The tail, in this view, is wagging the dog.

This view is supported by studies from the staff of the Permanent Subcommittee on Investigations (PSI) of the Senate Committee on Homeland Security and Government Affairs, which found that excessive speculation has had "undue" influence on wheat price movements and in the natural gas market. In the 2006 study of the impact of natural gas futures trading by the Amaranth hedge fund, the PSI found:

Amaranth's trading demonstrates that excessive speculation can distort futures prices not only in the next month or two, but for many months into the future. Currently, the major focus of the CFTC and the exchanges is to prevent excessive speculation from disrupting orderly trading of a contract near the expiration of that contract. The CFTC and the exchanges need to be vigilant to ensure that traders' speculative positions in futures contracts several seasons, or even several years, in advance are not distorting prices.

Also, a 2011 report by the minority staff of the House Committee on Oversight and Government Reform argues that "addressing excessive speculation offers the single most significant opportunity to reduce the price of gas for American consumers." Others, such as CFTC Commissioner Bart Chilton, have contended that oil price gyrations are likely the result of speculative trading. A frequent argument has been that a growing volume of investment flows from financial investors has affected prices. One econometric analysis that incorporated oil supply and energy demand effects concluded that speculation did not explain increases in oil prices in the 2003-2008 period, although the study suggested that speculation may have played some role in previous oil price spikes.

When financial institutions and investors as a group move funds into commodity markets, prices move. Even though increased financial speculation does not rise to the level of illegal manipulation, critics argue that the economic impact is the same.

In theory, higher volumes of speculative trading should not necessarily lead to more price volatility, if financial speculators base their trading decisions on the same factors as those of other market participants. But do they? The second part of the case that excessive speculation is destabilizing is that speculators do not trade on the fundamentals. The argument is that because financial speculators never produce or take delivery of physical oil, they bring to the market strategies and expectations that, in Keynes' phrase, "do not really make much difference to the prospective yield" of the asset. As a result, prices are subject to violent swings even though there has been no significant change in underlying supply and demand.

When oil prices are high, it is common to speak of a "speculative premium," meaning that the market price is higher than what the fundamentals of supply and demand justify, and that the excess is caused by uninformed speculation.

The CEO of ExxonMobil Corporation addressed this issue in testimony before the Senate Finance Committee in May 2011. Asked what the price of oil would be if it were based solely on fundamentals of supply and demand, he replied that (in terms of the marginal cost of producing the next barrel of oil), "it's going to be somewhere in the $60 to $70 range." But he also made more general comments on the role of speculation:

[I]t is very difficult to precisely say what impact [speculation] has, and it's also very difficult to separate in the marketplace speculation and risk management, because the two are actually quite intertwined in terms of how people manage the risk of the price of the fuel, whether they're a consumer or a producer.

I would give you just one benchmark. Immediately after the Libyan outbreak, the fighting that we have, within the next day the price of oil went up $12. Now, nothing had changed in the global supply the next day, so what was the market reacting to?

It was reacting to some level of insecurity about what the future supply was going to be. So that is people pricing into the global market what they believed their cost is going to be sometime in the future, building in their concerns and their worries about other possible supply disruptions and the ability of the market to respond to that.

In other words, possible future supply and demand events are properly factored into today's price, even though those events may never occur. Present-day supply and demand conditions are fundamentals, but so are expectations about the future. In general, free markets are expected to distinguish between relevant fundamental information and extraneous "noise" that causes prices to drift away from fundamental values.

The argument that speculation is distorting the oil market is based on one or both of two presumed mechanisms: (1) excessive speculation by financial traders is economically (if not legally) equivalent to price manipulation, and (2) speculators introduce unwarranted volatility by trading on information unrelated to fundamentals. The next section of this report briefly analyzes the fundamentals of the oil market and suggests that sharp swings in the price of oil do not necessarily mean that prices are not based on fundamentals.

Fundamental Factors

A number of factors have contributed to higher prices for oil and other energy commodities. Rapid global economic growth led to rising demand for oil, and supply could not keep up at previous prevailing oil prices. In theory, this contributes to prices rising until some consumers no longer buy oil and producers provide more supply, putting the market in balance again. But oil supply and demand is inelastic to price changes, especially in the near-term, which means it may take a larger percentage change in prices to incentivize relatively small changes in supply or demand.

Economic Growth Lifts Demand; Supply Cannot Keep Up

Global economic growth increased demand for oil. Economic growth is the leading driver of oil demand. It accelerated to an average of nearly 5% between 2002 and 2007—the fastest five years of global economic growth on record except for the five years preceding the oil price increases of the 1970s. Much of this growth took place in emerging market and developing countries, not advanced economies such as the United States or Europe. (See Figure 7 .) These countries were going through the energy-intensive process of industrialization, which required greater use of energy sources such as oil and coal.

Oil supply growth, on the other hand, faced a number of hurdles. Oil resources in key oil exporting countries like Mexico, the United Kingdom, and Norway had been depleted and were in decline. Other key sources of oil supply experienced supply disruptions that reduced production. Examples included strikes in Venezuela in 2003, periodic militant attacks in Nigeria (particularly since 2003), and hurricanes in the Gulf of Mexico sometimes shutting down offshore U.S. and Mexican production.

Further, some countries with abundant oil resources maintained or raised new barriers to private investment in oil exploration and production, such as increasing the national oil company's control of the energy sector, raising industry taxes, or effectively nationalizing shares in some oil producing assets.

The Organization of Petroleum Exporting Countries (OPEC) played a special role among producers. This cartel of oil exporting countries varies its production level in an attempt to control oil prices. OPEC countries thus collectively maintain spare oil production capacity that can act as a cushion to oil markets in the event of disruptions or other surprising developments that require more oil. (This contrasts from production in non-OPEC countries, which is usually at maximum capacity.) OPEC did cut production at several key points when prices were falling during the 2000s. But generally OPEC members increased their output over the period when prices were rising. Perhaps more importantly, the organization ran low on spare capacity between 2004 and 2008. Low spare oil production capacity reduces the capacity to cushion against future supply disruptions or other market surprises. OPEC countries hold 77% of the world's known oil reserves, but they produce only about 42% of the world's oil supply. By failing to develop more of these extensive reserves and either supply oil or establish spare production capacity, OPEC members arguably contributed to the rise in oil prices in recent years.

Price Inelasticity of Supply and Demand Enables Large Price Swings

A rising price for a good provides consumers with an incentive to consume less of that good and provides producers an incentive to supply more, which in turn can moderate the price increase. But in the oil market, supply and demand quantities can be relatively unresponsive to price changes, especially in the short run. As a result, it takes large swings in price to correspond to relatively small changes in consumption and production.

Oil is essential for economic activity and there are limited near-term substitutes. Consequently, households and industrial consumers may absorb much of the cost increases, reducing spending on other goods or reducing savings. Rapid economic growth in developing countries meant rising incomes could absorb higher energy costs. Further, some countries had subsidies that insulated consumers from the price increase during the 2000s. (Some developing countries have subsequently reformed their pricing system to reduce the fiscal burden of subsidies and reduce consumption.)

In advanced economies, incomes were already relatively high, allowing consumers to absorb higher costs for a time, albeit at the expense of other economic priorities—a painful adjustment particularly for low-income households as well as businesses and workers in industries where oil-related expenditure is a relatively significant part of the budget. Consumers could not easily reduce their consumption through efficiency improvements—the equipment that uses oil is expensive and upgrading or replacing it can require large upfront costs and take time. Examples include cars, home heating, airplanes, and industrial equipment.

These factors all contribute to global demand that is inelastic to prices: consumption did not decline in proportion to the increase in prices. In fact, except for 2008 and 2009—when the recession dragged down global demand—oil consumption has increased every year since 1993. Global oil consumption increased by roughly the same amount in the decade of the 2000s as it did during the 1990s, despite oil prices being at much higher levels. Consumption in mid-2011 has recovered from the recession, surpassing previous highs to reach record levels. Underlying these developments is a shift in consumption growth from the advanced economies to developing economies (see Figure 8 ).

Oil supply is also slow to respond. Oil production assets are expensive, and developing large new fields can take many years. As a result, supply can also be inelastic to prices in the short run. This tendency is exacerbated by several factors. As easy-to-develop resources have been depleted, the oil industry has moved into resources that are more complicated, expensive, and difficult to exploit, such as the oil sands in Canada or deepwater offshore developments. Where abundant easy-to-develop resources are still available, countries sometimes restrict where and how oil development and production can take place in pursuit of other national objectives, including environmental and resource management priorities. Nations may limit access to oil resources to preserve them for future generations, maintain government control of the energy industry through a national oil company, or maximize long-term revenues from energy resources.

Alternatives to oil, like biofuels, electric or gas vehicles, and coal- or gas-to-liquids technology have also faced challenges that have made them difficult to scale up. Some are expensive, require significant long term investment in infrastructure, lack sufficient technical advances, or have other potential negative impacts (consider ethanol contributing to higher food prices or coal-to-liquids emitting significant greenhouse gases). Even with higher prices, many of these technologies still required public support and were thus subject to policy and political uncertainties and constraints.

With supply and demand adjustment constrained, there was little to dampen the price increase, unlike what might take place in a more price-elastic market. However, demand is relatively elastic to income. When the U.S. economic recession spread to the world in the second half of 2008, falling incomes quickly reduced demand, and prices followed. Again because supply is inelastic to price movements, few producers curtailed supply when prices were falling. Even when oil prices reached near $30 per barrel, it remained above the operating cost requirements of nearly all sources of oil supply, so producers did not have to shut down for commercial reasons. In the event, OPEC responded by making a policy decision to curtail output to support prices. These cuts reached markets in early 2009. Cuts, along with the economic recovery, contributed to subsequent price increases. In recent months, geopolitical disruptions again came to the forefront of the market. Supply disruptions and fears of future disruptions in the Middle East and North Africa, coupled with economic recovery and the persistence of supply constraints, have contributed to oil prices reaching levels seen in the first half of 2008.

Congressional Action

Legislation before the 112 th Congress ( S. 1200 and H.R. 2328 , both entitled End Excessive Oil Speculation Now Act of 2011) would authorize and direct the CFTC to take certain actions to reduce the volume of speculation in oil and related energy commodities. These identical bills would impose a margin requirement of 12% for swaps and futures traded on crude oil, gasoline, diesel fuel, jet fuel, and heating oil. Such margin requirements would not apply, however, to "bona fide hedging" transactions, including those that represent a substitute for a position to be taken at a later time in a physical marketing channel, and those used to hedge a potential change in value of assets held or produced.

S. 1200 and H.R. 2328 also mandate that the CFTC impose speculative position limits on swaps and futures in crude oil, gasoline, diesel fuel, jet fuel, or heating oil that are equal to the position accountability levels or position limits established by the New York Mercantile Exchange (Nymex). The bills include a sunset provision by which they would be terminated once the CFTC imposes position limits as required by Title VII of the Dodd-Frank Act.

H.R. 2003 , the Taxing Speculators Out of the Oil Market Act, would impose a tax on oil futures, swaps, and options transactions, except for those hedging commercial risk. The tax would be levied at 0.01% of the value of a futures contract; 0.01% of the premium paid on an option; or, for a swap, 0.01% of the value of the underlying assets for each year until the swap matures.

Conclusion

Supply and demand issues—market fundamentals—played a central role in the increase of oil prices in recent years. Fundamental factors may have also created the conditions that enabled financial factors to have an impact on price: in a price-elastic market, purely financially driven price run-ups could quickly set off supply and demand adjustments that could again bring prices down. The absence of such adjustments may allow financial investors to drive prices for periods of time. Conditions in financial markets in energy contracts may also exacerbate volatility: relatively small changes in speculative positions appear to be associated at times with significant price swings. What remains unresolved is how much price movement in recent years is attributable to fundamental factors versus financial factors.

Appendix A. Speculative Trading and Price

Table A-1 shows the results of a linear regression analysis conducted using the software program STATA. The results show a statistically significant positive correlation between the net weekly change in managed money positions (i.e., the net long or short positions), and the weekly change in oil prices. The regression uses the same weekly data from the CFTC discussed in this report. The correlation between the weekly oil price change and the net positions of hedgers, swap dealers and "other" traders, however, is not statistically significant, using a 95% confidence interval. In interpreting the results, the coefficient ("Coef." in column 2) shows the marginal, or incremental, effect of the independent variables on the dependent variable. Here, the independent variables are the weekly net changes in position for each of the four groups of traders; and the dependent variable is the weekly change in the price of oil. In the case of money managers, for instance, the coefficient of .7635849 implies that a net change of 1,000 positions by managed money traders would be associated with a change in the price of oil of about 76 cents per barrel. The complete dataset had 125 observations, or data points.

CRS Report R41902, Hedge Fund Speculation and Oil Prices , by [author name scrubbed] and [author name scrubbed], finds that a similar statistical relationship holds when COT report data are analyzed, and that the correlation is robust after controlling for certain macroeconomic variables.

Appendix B. Options

In the futures contracts discussed in the text, all gains by short traders create equal losses by long traders (or vice versa): futures trading is a zero-sum game. Traders who wish to limit their potential losses may choose to employ options, where gains and losses are not symmetrical. The key distinction between options and futures is that one party has the right, but not the obligation, to buy an asset in the future at a price determined when the option is purchased. There are two kinds of options: calls and puts. A call gives the holder of the options contract the right to buy an asset at a fixed price, whereas a put gives the holder the right to sell at a fixed price.

The price at which the underlying asset may be bought or sold is called the exercise price, or the strike price. An options contract confers the right to buy or sell for a specified period of time—each option has an expiration date.

On the other side of a put or call is the seller, or writer, of the option. The seller is obliged to buy or sell the asset at the strike price whenever the buyer chooses to exercise the option. In exchange for this right, the seller of the option receives a one-time payment, called the premium. The buyer's risk is limited to the amount of the premium—if prices move contrary to what the buyer expected, he simply lets the option expire unexercised, and the seller keeps the premium. On the other hand, the option buyer's potential profit is unlimited (just as a futures trader's is), because no matter how high or low the market price of the underlying asset may go, the option writer is obliged to buy or sell at the specified strike price.

The price of an option is reflected in the amount of the premium that is charged by the seller. A number of factors affect option prices: first, the relationship between the strike price and the current market price of the asset, which is called the intrinsic value of the option. If, for example, a put option on 100 shares of Company A's stock has a strike price of $14 and the current share price is $13.50, the intrinsic value of the contract to the buyer is $50 ($0.50 per share times 100 shares). An option is said to be "in the money" when the holder can exercise at a profit. If Company A's shares climbed to $15, the put option would be "out of the money," or "underwater," because the right to sell a $15 share for $14 is worthless.

In addition to intrinsic value, an option has time value. If the put on Company A in the example above is currently out of the money, there is still the chance that the share price will drop below the strike price before the option expires. Time value depends on the length of time to expiration and the price volatility of the underlying asset, which determines the probability of the option coming into the money during the life of the contract.

Options are traded both on securities and futures exchanges and over-the-counter (OTC). Underlying assets include stocks, stock indexes, futures contracts, currencies, interest rates, and physical commodities. Many OTC contracts include option-like features, including swaps, which are discussed in Appendix C .

Appendix C. Swaps

Counterparties to a swap contract agree to exchange payments over a specified time period. In the simplest form, one payment is fixed, while the other fluctuates in accordance with changes in some financial variable, such as an interest rate, a stock index, or a commodity price.

In a simple oil swap, one counterparty might agree to buy 1 million barrels of crude oil from the other every three months over the next five years for $80/barrel. The other counterparty would agree to buy 1 million barrels at the Nymex spot-month price on the day the payment was due. Counterparty 1, who might be an airline wishing to hedge the risk of fuel price increases, has locked in the $80 price for five years. It will gain if the market price rises above $80.

Counterparty 2 is committed to selling oil at $80/barrel—it will benefit from the swap if current market prices drop below $80. It is possible that the second counterparty is hedging the risk of falling prices, as a producer would wish to do, but in practice most swaps involve a dealer who is willing to offer swaps on either side of the market (that is, take the floating or fixed leg of the swap). Swap dealers may manage the risk of their price exposure through another offsetting swap, or they may take an offsetting position in futures markets. (Assuming that the second counterparty in this hypothetical oil swap is a dealer, it would take a long position in futures, which would pay off if the price of oil increased. Gains in futures would offset losses on the swap; the dealer would profit through spreads and fees.)

In practice, the swapping of fixed for floating payments does not occur. The counterparties observe the change in the price of oil since the last swap payment date and calculate a single net payment, which actually changes hands.

Swaps generally do not require physical delivery of oil. Contracts in which delivery is mandatory are considered to be forward contracts and are exempt from regulation under the Commodity Exchange Act.