Please cite this paper as:

Irwin, S. H. and D. R. Sanders (2010), “The Impact of

Index and Swap Funds on Commodity Futures Markets:

Preliminary Results”, OECD Food, Agriculture and

Fisheries Working Papers, No. 27, OECD Publishing.

doi: 10.1787/5kmd40wl1t5f-en

OECD Food, Agriculture and Fisheries

Working Papers No. 27

The Impact of Index and

Swap Funds on Commodity

Futures Markets

PRELIMINARY RESULTS

Scott H. Irwin

*

, Dwight R. Sanders

*University of Illinois, United States
TAD/CA/APM/WP(2010)8/FINAL

1

Executive Summary

The report was prepared for the OECD by Professors Scott Irwin and Dwight Sanders. It represents a 

preliminary study which aims to clarify the role of index and swap funds in agricultural and energy 

commodity futures markets. The full report including the econometric analysis is available in the Annex to 

this report.

While the increased participation of index fund investments in commodity markets represents a 

significant structural change, this has not generated increased price volatility, implied or realised, in 

agricultural futures markets. Based on new data and empirical analysis, the study finds that index funds did 

not cause a bubble in commodity futures prices. There is no statistically significant relationship indicating 

that changes in index and swap fund positions have increased market volatility. The evidence presented 

here is strongest for the agricultural futures markets because the data on index trader positions are 

measured with reasonable accuracy. The evidence is not as strong in the two energy markets studied here 

because of considerable uncertainty about the degree to which the available data actually reflect index 

trader positions in these markets. 

An unexpected finding was a negative relationship between index and swap fund positions and market 

volatility. That is, there is some evidence that increases in index trader positions are followed by lower 

market volatility. This result must be interpreted with considerable caution. The possibility still exists that 

trader positions are correlated with some third variable that is actually causing market volatility to decline. 

Nonetheless, this finding is contrary to popular notions about the market impact of index funds, but is not 

so surprising in light of the traditional problem in commodity futures markets of the lack of sufficient 

liquidity to meet hedging needs and to transfer risk.

The empirical evidence presented in this preliminary study does not appear at present to warrant 

extensive changes in the regulation of index funds participation in agricultural commodity markets; any 

such changes require careful consideration so as to avoid unintended negative impacts. For example,

limiting the participation of index fund investors could unintentionally deprive commodity futures markets 

of an important source of liquidity and risk-absorption capacity at times when both are in high demand.

Lack of convergence between spot and futures prices in certain markets, however, does raise a 

number of issues about the functioning of these markets and possible role of index funds. Further research 

is needed to understand better these recent structural changes in futures marks and how they may impact on 

the dynamics of price formation. But at this time, the weight of evidence clearly suggests that increased 

index fund activity in 2006-08 did not cause a bubble in commodity futures prices.
TAD/CA/APM/WP(2010)8/FINAL

2

TABLE OF CONTENTS

THE IMPACT OF INDEX AND SWAP FUNDS ON COMMODITY FUTURES MARKETS: 

PRELIMINARY RESULTS ...........................................................................................................................3

1. Introduction..............................................................................................................................................3

2. It was a Bubble ........................................................................................................................................6

3. It was not a Bubble ..................................................................................................................................7

4. Evidence to date.....................................................................................................................................10

5. New evidence.........................................................................................................................................11

6. Policy Conclusions.................................................................................................................................21

REFERENCES..............................................................................................................................................23

GLOSSARY OF TERMS..............................................................................................................................25

Tables

Table 1. Causal relationships estimated for market system, June 2006 - December 2009 ........................14

Table 2. Summary statistics, net long positions held by index traders and swap dealers

(# of contracts) June 2006-December 2009 ..............................................................................................16

Table 3. Percent of total open interest held by CIT and DCOT categories, June 2006-December 2009...17

Table 4. Granger causality test results for CIT net positions do not lead returns, June 2006-December 

2009 ...........................................................................................................................................................18

Table 5. Granger causality test results for DCOT swap dealer net positions do not lead realized volatility, 

June 2006-December 2009 ........................................................................................................................19

Table 6. Summary statistics, working's speculative T-Index, adjusted for index trader positions, June 

2006- December 2009................................................................................................................................20

Table 7. Granger causality test results for T-Index does not lead realized volatility, June 2006-December 

2009 ...........................................................................................................................................................21

Figures

Figure 1. Commodity index fund investment (year end), 1990 – 2009 .......................................................4

Figure 2. CRB Commodity index, January 2006 - September 2009 ...........................................................4

Figure 3. Hypothetical example of a convex pricing function for a storable commodity............................9

Figure 4. Index trader net long positions in CBOT wheat and nearby futures prices, June 2006-December 

2009 ...........................................................................................................................................................12

Figure 5. Contemporaneous relationship, CBOT wheat returns (price change) and index trader net long 

positions, June 2006-December 2009 ........................................................................................................13

Figure 6. Causal relationship, CBOT wheat returns (price change) and index trader net long positions, 

June 2006-December 2009 ........................................................................................................................14
TAD/CA/APM/WP(2010)8/FINAL

3

THE IMPACT OF INDEX AND SWAP FUNDS ON COMMODITY FUTURES MARKETS:

PRELIMINARY RESULTS

1. Introduction

1. The financial industry has developed new products that allow institutions and individuals to 

invest in commodities through long-only index funds, over-the-counter (OTC) swap agreements, exchange 

traded funds, and other structured products.1 Box 1 provides key definitions used in the discussion; see the 

glossary for a complete set of definitions. Regardless of form, these instruments have a common goal: to 

provide investors with buy-side exposure to returns from a particular index of commodity prices. The S&P 

GSCI IndexTM (Standard‘s and Poor‘s Goldman Sachs Commodity Index) is one of the most widely 

tracked indexes and is generally considered an industry benchmark. It is computed as a productionweighted average of the prices from 24 commodity futures markets. 

2. Several influential studies in recent years purport that investors can capture substantial risk 

premiums and reduce portfolio risk through relatively modest investment in long-only commodity index 

funds. Combined with the availability of deep and liquid exchange-traded futures contracts, this evidence 

fuelled a dramatic surge in index fund investment. Some describe this surge and its attendant impacts as 

the ―financialization‖ of commodity futures markets. Given the size and scope of commodity index funds,

it should probably not come as a surprise that a world-wide debate has ensued about their role in 

commodity markets. The debate has important ramifications from a policy and regulatory perspective as 

well as practical implications for the efficient pricing of commodity products. 

3. There are a few indisputable facts about the behaviour of commodity futures markets over 2006-

08, the period associated with the most controversy regarding the impact of money inflows from 

commodity index funds. First, inflows into long-only commodity index funds did increase rather 

substantially throughout 2006-08 (see Figure 1). According to the most widely-quoted industry source 

(Barclays) index fund investment increased from USD 90 billion at the beginning of 2006 to a peak of just 

under USD 200 billion at the end of 2007. Second, commodity prices have also increased rather 

dramatically - 71% as measured by the Commodity Research Bureau index - from January 2006 through 

June of 2008 (see Figure 2). Third, prices declined almost equally dramatically from June 2008 through 

early 2009 (see Figure 2). These facts are clear and not in dispute. It‘s the interpretation of the interaction 

among these facts that is so controversial.

 

1

In the remainder of this report, the term ―commodity index fund‖ or ―index fund‖ is used generically to 

refer to all of the varied long-only commodity investment instruments. 
TAD/CA/APM/WP(2010)8/FINAL

4

Figure 1. Commodity index fund investment (year end), 1990 – 2009

Figure 2. CRB Commodity index, January 2006 - September 2009

150

170

190

210

230

250

270

Jan-06

Apr-06

Jul-06

Oct-06

Jan-07

Apr-07

Jul-07

Oct-07

Jan-08

Apr-08

Jul-08

Oct-08

Jan-09

Apr-09

Jul-09

Index

Month

4. On one side, some hedge fund managers, commodity end-users, and policy-makers assert that 

speculative buying by index funds on such a wide scale created a ―bubble,‖ with the result that commodity 

futures prices far exceeded fundamental values during the boom. This view has led to new regulatory 

initiatives to limit speculative positions in commodity futures markets. On the other side, a number of 

economists have expressed scepticism about the bubble argument. These economists argue that commodity 

markets were driven by fundamental factors that pushed prices higher. For example, the main factors cited 

as driving the price of crude oil include strong demand from China, India, and other developing nations, a 

levelling out of crude oil production, a decrease in the responsiveness of consumers to price increases, and 

U.S. monetary policy. In the grain markets, the diversion of row crops to biofuel production and weather0

50

100

150

200

250

300

1990 1992 1994 1996 1998 2000 2002 2004 2006 2008

Investment (bil. $)

Source: Barclays
TAD/CA/APM/WP(2010)8/FINAL

5

related production shortfalls are cited, as well as demand growth from developing nations and U.S. 

monetary policy. 

Box 1. Key definitions

Speculator: In commodity futures, a trader who does not hedge, but who trades with the objective of achieving 

profits through the successful anticipation of price movements

Hedger: A trader who enters into positions in a futures market opposite to positions held in the cash market to 

minimize the risk of financial loss from an adverse price change; or who purchases or sells futures as a 

temporary substitute for a cash transaction that will occur later. One can hedge either a long cash market position 

(e.g., one owns the cash commodity) or a short cash market position (e.g., one plans on buying the cash 

commodity in the future). 

Swap: In general, the exchange of one asset or liability for a similar asset or liability for the purpose of 

lengthening or shortening maturities, or otherwise shifting risks. This may entail selling one securities issue and 

buying another in foreign currency; it may entail buying a currency on the spot market and simultaneously selling 

it forward. Swaps also may involve exchanging income flows; for example, exchanging the fixed rate coupon 

stream of a bond for a variable rate payment stream, or vice versa, while not swapping the principal component 

of the bond. Swaps are generally traded over-the-counter. 

Swap Dealer (AS): An entity such as a bank or investment bank that markets swaps to end users. Swap dealers 

often hedge their swap positions in futures markets. 

Commodity Index Funds: Financial product whose value is based on an index of commodity futures prices.

Over-the-Counter (OTC): The trading of commodities, contracts, or other instruments not listed on any 

exchange. OTC transactions can occur electronically or over the telephone.

Speculative Bubble: A rapid run-up in prices caused by excessive buying that is unrelated to any of the basic, 

underlying factors affecting the supply or demand for a commodity or other asset. Speculative bubbles are 

usually associated with a "bandwagon" effect in which speculators rush to buy the commodity (in the case of 

futures, "to take positions") before the price trend ends, and an even greater rush to sell the commodity (unwind 

positions) when prices reverse. 

Long: (1) One who has bought a futures contract to establish a market position; (2) a market position that 

obligates the holder to take delivery; (3) one who owns an inventory of commodities. 

Long Hedge: Hedging transaction in which futures contracts are bought to protect against possible increases in 

the cost of commodities. 

Short: (1) The selling side of an open futures contract; (2) a trader whose net position in the futures market 

shows an excess of open sales over open purchases. See Long. 

Short Hedge: Selling futures contracts to protect against possible decreased prices of commodities. 

Open Interest: The total number of futures contracts long or short in a delivery month or market that has been 

entered into and not yet liquidated by an offsetting transaction or fulfilled by delivery.

Excessive Speculation: Amount of speculation beyond that which is necessary or normal relative to hedging 

needs, as measured by Working’s T. A large part of technically excess speculation is economically necessary for 

a well functioning market. The ratio of the amount of speculation to hedging needs must thus be greater than 1

for futures markets to have sufficient liquidity to fulfill their economic role. For Working’s T-values of 1.15 or less 

markets are considered to have insufficient liquidity though there is an excess of speculation, technically 

speaking.

5. Even though almost two years have passed since the 2008 peak in commodity prices, the 

controversy surrounding index funds continues unabated. We contend that a detailed and dispassionate 

synthesis of the arguments and latest research will be of great utility to market observers and policymakers 
TAD/CA/APM/WP(2010)8/FINAL

6

given the strident nature of the debate. Policy makers need to have a full picture of the current state of 

scientific knowledge on the impact of commodity index funds before imposing costly new regulations. In 

this paper, we provide an overview of the arguments concerning the impact of index funds in commodity 

futures markets as well as an assessment of the latest research on the subject. We also summarise some 

new empirical evidence on the market impact of commodity index funds.

2. It was a Bubble

6. Masters (2008) has interwoven investment and price data to create the most widely-cited bubble 

argument, painting the activity of index funds as akin to the infamous Hunt brothers‘ cornering of the 

silver market. He blames the rapid increase in overall commodity prices from 2006-08 on institutional 

investors‘ embrace of commodities as an investable asset class. As noted in the introduction, it is clear that 

considerable dollars flowed into commodity index funds over this time period. However, the evidence 

provided by Masters is limited to anecdotes and the temporal correlation between money flows and prices. 

Masters and White (2008) recommend specific regulatory steps to address the alleged problems created by 

index fund investment in commodity futures markets, including the re-establishment of speculative 

position limits for all speculators in all commodity futures markets and the elimination or severe restriction 

of index speculation. 

7. A similar position was taken by the U.S. Senate Permanent Subcommittee on Investigations in its 

examination of the performance of the Chicago Board of Trade‘s (CBOT) wheat futures contract 

(USS/PSI, 2009, p. 2):

―This Report finds that there is significant and persuasive evidence to conclude that these 

commodity index traders, in the aggregate, were one of the major causes of ―unwarranted 

changes‖—here, increases—in the price of wheat futures contracts relative to the price of wheat in 

the cash market. The resulting unusual, persistent and large disparities between wheat futures and 

cash prices impaired the ability of participants in the grain market to use the futures market to price 

their crops and hedge their price risks over time, and therefore constituted an undue burden on 

interstate commerce. Accordingly, the Report finds that the activities of commodity index traders, 

in the aggregate, constituted ―excessive speculation‖ in the wheat market under the Commodity 

Exchange Act.‖ 

8. Based on these findings, the Subcommittee recommended: 1) phasing out of existing position 

limit waivers for index traders in wheat, 2) if necessary, imposition of additional restrictions on index 

traders, such as a position limit of 5 000 contracts per trader, 3) investigation of index trading in other 

agricultural markets, and 4) strengthening of data collection on index trading in non-agricultural markets.

9. One of the limitations of the bubble argument made by Masters and others is that the link 

between money inflows from index funds and commodity futures prices is not well developed. This allows 

critics to assert that bubble proponents make the classical statistical mistake of confusing correlation with 

causation. In other words, simply observing that large investments have flowed into the long side of 

commodity futures markets at the same time that prices have risen substantially does not necessarily prove 

anything without a logical and causal link between the two. One attempt to establish this linkage is found 

in Petzel‘s (2009, pp. 8-9) testimony at a CFTC hearing on position limits in energy futures markets:

―Seasoned observers of commodity markets know that as non‐commercial participants enter a 

market, the opposite side is usually taken by a short‐term liquidity provider, but the ultimate 

counterparty is likely to be a commercial. In the case of commodity index buyers, evidence 

suggests that the sellers are not typically other investors or leveraged speculators. Instead, they are 

owners of the physical commodity who are willing to sell into the futures market and either deliver 
TAD/CA/APM/WP(2010)8/FINAL

7

at expiration or roll their hedge forward if the spread allows them to profit from continued storage. 

This activity is effectively creating ―synthetic‖ long positions in the commodity for the index 

investor, matched against real inventories held by the shorts. We have seen high spot prices along 

with large inventories and strong positive carry relationships as a result of the expanded index 

activity over the last few years.‖

10. In essence, Petzel argues that unleveraged futures positions of index funds are effectively 

synthetic long positions in physical commodities, and hence represent new demand. If the magnitude of 

index fund demand is large enough relative to physically-constrained supplies in the short-run, prices and 

price volatility can increase sharply. The bottom-line is that the size of index fund investment is ―too big‖ 

for the current size of commodity futures markets.

11. Hamilton (2009) provides a more formal theoretical treatment of the issues. He begins by noting 

that the key challenge is reconciling a speculative bubble in crude oil prices with changes in the physical 

quantities of crude oil. A standard argument is that a price bubble will inevitably lead to a rise in 

inventories as the quantity supplied at the ―bubble price‖ exceeds the quantity demanded. Hamilton‘s 

theoretical model shows the conditions that must occur for index fund speculation to lead to a bubble in a 

storable commodity market such as crude oil. First, index fund positions in the futures market must have a 

positive relationship to the level of futures prices. Otherwise there is no mechanism for the flow of index 

fund investment to initiate the bubble that starts in the futures market. Second, the elasticity of demand for 

the commodity (or the final product, gasoline in the case of crude oil) must be zero or very close to zero.

This allows the bubble-related increase in the futures price to be fully passed on to consumers. Third, 

inventories of the commodity must not increase. These conditions provide an important theoretical 

framework on which to base empirical tests for the potential of price bubbles in storable commodity 

futures prices.

3. It was not a Bubble

12. A number of economists have expressed scepticism about the bubble argument. These 

economists cite several contrary facts and argue that commodity markets were driven by fundamental 

factors that pushed prices higher. Irwin, Sanders, and Merrin (2009) present a useful summary of the 

counter arguments made by these economists. Specifically, they note three logical inconsistencies in the 

arguments made by bubble proponents as well as five instances where the bubble story is not consistent 

with observed facts. Here, we review these points as well as some additional arguments made by both proand anti-bubble proponents in response. 

13. The first possible logical inconsistency within the bubble argument is equating money inflows to 

commodity futures markets with demand. With equally informed market participants, there is no limit to 

the number of futures contracts that can be created at a given price level. Index fund buying in this 

situation is no more ―new demand‖ than the corresponding selling is ―new supply‖. Combined with the 

observation that commodity futures markets are zero-sum games, this implies that money flows in and of 

themselves do not necessarily impact prices. Prices will only change if new information emerges that 

causes market participants to revise their estimates of physical supply and/or demand. 

14. What happens when market participants are not equally informed? When this is the case, it is 

rational for participants to condition demands on both their own information and information about other 

participants‘ demands that can be inferred (―inverted‖) from the futures price. The trades of uninformed 

participants can impact prices in this more realistic model if informed traders mistakenly believe that trades 

by uninformed participants reflect valuable information. Hence, it is possible that other traders in 

commodity futures markets interpreted the large order flow of index funds on the long side of the market 

as a reflection of valuable private information about commodity price prospects, which would have had the 
TAD/CA/APM/WP(2010)8/FINAL

8

effect of driving prices higher as these traders revised their own demands upward. Of course, this would 

have required a large number of sophisticated and experienced traders in commodity futures markets to 

reach a conclusion that index fund investors possessed valuable information that they themselves did not 

possess.

15. The second possible logical inconsistency is to argue that index fund investors artificially raised 

both futures and cash commodity prices when they only participated in futures markets. Futures contracts 

are financial transactions that only rarely involve the actual delivery of physical commodities. In order to 

impact the equilibrium price of commodities in the cash market, index investors would have to take 

delivery and/or buy quantities in the cash market and hold these inventories off the market. Index investors 

are purely involved in a financial transaction using futures markets; they do not engage in the purchase or 

hoarding of the cash commodity and any causal linkages between their futures market activity and cash 

prices is unclear at best. Hence, it is wrong to draw a parallel between index fund positions and past efforts 

to ―corner‖ commodity markets, such as the Hunt brothers' effort to manipulate the silver market in 1979-

80. 

16. A third possible logical inconsistency is a blanket categorization of speculators, in particular, 

index funds, as wrongdoers and hedgers as victims of their actions. In reality, the ―bad guy‖ is not so easily

identified since hedgers sometimes speculate and some speculators also hedge. For example, large 

commercial firms may have valuable information gleaned from their far-flung cash market operations and 

trade based on that information. The following passage from a recent article on Cargill, Inc. (Davis, 2009) 

nicely illustrates the point:

Wearing multiple hats gives Cargill an unusually detailed view of the industries it bets on, as well 

as the ability to trade on its knowledge in ways few others can match. Cargill freely acknowledges 

it strives to profit from that information. "When we do a good job of assimilating all those 

seemingly unrelated facts," says Greg Page, Cargill's chief executive, in a rare interview, "it 

provides us an opportunity to make money...without necessarily having to make directional trades, 

i.e., outguess the weather, outguess individual governments."

17. The implication is that the interplay between varied market participants is more complex than a 

standard textbook description of pure risk-avoiding hedgers and pure risk-seeking speculators. The reality 

is that market dynamics are ever changing and it can be difficult to understand the motivations and market 

implications of trading, especially in real-time. 

18. In addition to the logical inconsistencies, there are several ways the bubble story is not consistent 

with the observed facts. First, as Krugman (2008) asserts, if a bubble raises the market price of a storable 

commodity above the true equilibrium price, then stocks of that commodity should increase (much like a 

government imposed price floor can create a surplus). Stocks were declining, not building, in most 

commodity markets over 2006-08, which is inconsistent with the depiction of a price bubble in these 

markets.

19. Second, the relationship between prices and inventories for storable commodities is highly 

convex. Figure 3, drawn from Wright (2009), illustrates this point. Note that a given reduction in quantity 

due a to supply and/or demand shock will have a much larger impact on price when starting with a low 

quantity (inventories) compared to when starting with a high quantity. It also implies that relatively minor 

reductions in quantity can result in very large increases in price when the market supply/demand balance is 

especially tight. Smith (2009) argues that it is plausible that a series of seemingly small supply disruptions 

in the spring and summer of 2008 could explain the large increase in crude oil prices during this time 

period in view of the extreme convexity of the pricing function for crude oil in the short-run. 
TAD/CA/APM/WP(2010)8/FINAL

9

Figure 3. Hypothetical example of a convex pricing function for a storable commodity

20. Third, theoretical models that show uninformed or noise traders impacting market prices rely on 

the unpredictable trading patterns of these traders to make arbitrage risky. Because the arbitrage - needed 

to drive prices to fundamental value - is not riskless, noise traders can drive a wedge between market prices 

and fundamental values. Importantly, index fund buying is very predictable. That is, index funds widely 

publish their portfolio (market) weights and roll-over periods. Thus, it seems highly unlikely that other 

large and rational traders would hesitate to trade against an index fund if they were driving prices away 

from fundamental values.

21. Fourth, if index fund buying drove commodity prices higher then markets without index fund 

investment should not have seen prices advance. Again, the observed facts are inconsistent with this 

notion. Irwin, Sanders, Merrin (2009) show that markets without index fund participation (fluid milk and 

rice futures) and commodities without futures markets (apples and edible beans) also showed price 

increases over the 2006-2008 period. Stoll and Whaley (2009) report that returns for Chicago Board of 

Trade (CBOT) wheat, Kansas City Board of Trade (KCBOT) wheat, and Minneapolis Grain Exchange 

(MGEX) wheat are all highly positively correlated over 2006-09, yet only CBOT wheat is used heavily by 

index investors. In a similar fashion, Commodity Exchange (COMEX) gold, COMEX silver, New York 

Mercantile (NYMEX) palladium, and NYMEX platinum futures prices are highly correlated over the same 

time period but only gold and silver are included in popular commodity indexes. Headey and Fan (2008) 

cite the rapid increases in the prices for ―non-financialized‖ commodities such as rubber, onions, and iron 

ore as evidence that rapid price inflation occurred in commodities without futures markets. While certainly 

instructive, the limits of these kinds of comparisons also need to be kept in mind. Bubble proponents have 

pointed out that commodity markets selected for the development of futures contracts may be naturally 

more volatile than those commodities without futures markets.
TAD/CA/APM/WP(2010)8/FINAL

10

22. Fifth, speculation was not excessive when correctly compared to hedging demands. The statistics 

on long-only index fund trading reported in the media and discussed at hearings tend to view speculation in 

a vacuum - focusing on absolute position size and activity. Working (1960) argued that speculation must 

be gauged relative to hedging needs. In particular, speculation can only be considered  ̳excessive‘ relative 

to the level of hedging activity in the market. Utilizing Working‘s speculative ―T-index‖, Sanders, Irwin, 

and Merrin (2010) demonstrate that the level of speculation in nine agricultural futures markets from 2006-

08 (adjusting for index fund positions) was not excessive. Indeed, the levels of speculation in all markets 

examined were within the realm of historical norms. Across most markets, the rise in index buying was 

more than offset by commercial (hedger) selling. Buyuksahin and Harris (2009) use daily data from the 

CFTC‘s internal large trader database to show that Working‘s T-index in the crude oil futures market 

increased in parallel with crude oil prices over 2004-09 but the peak of the index was still well within 

historical norms. Till (2009) reports similar results for crude oil, heating oil, and gasoline futures over 

2006-2009 using recently available data in the CFTC‘s Disaggregated Commitments of Traders report. 

23. The sixth observable fact revolves around the impact of index funds across markets. A priori, 

there is no reason to expect index funds to have a differential impact across markets given similar position 

sizes. That is, if index funds can inflate prices, they should have a uniform impact across markets for the 

same relative position size. It is therefore difficult to rationalize why index fund speculation would impact 

one market but not another. Further, one would expect markets with the highest concentration of index 

fund positions to show the largest price increases. Irwin, Sanders, and Merrin (2009) find just the opposite 

when comparing grain and livestock futures markets. The highest concentration of index fund positions 

was often in livestock markets, which had smallest price increases through the spring of 2008. This is 

difficult to reconcile with the assertion that index buying represents demand.

4. Evidence to date

24. Not surprisingly, a flurry of studies has been completed recently in an attempt to sort out which

side of the debate is correct. Some studies find evidence that commodity index funds have impacted 

commodity futures prices (Gilbert, 2009; Einloth, 2009; Tang and Xiong, 2010). Results in these studies 

negate the argument that no evidence exists of a relationship between index fund trading and movements in 

commodity futures prices. However, the evidence is weak because the data and methods used in most of 

these studies are subject to a number of important criticisms. Hamilton‘s (2009) study, while not definitive 

in terms of empirics, is the most important of this group because his theoretical model shows the 

conditions that must occur for index fund speculation to lead to bubble impacts in a storable commodity 

market such as crude oil.

25. A number of studies find little evidence of a relationship between index fund positions and 

movements in commodity futures prices (Stoll and Whaley, 2009; Buyuksahin and Harris, 2009; Sanders 

and Irwin, 2010a, 2010b; Aulerich, Irwin, and Garcia, 2010). This constitutes a rejection of the first 

theoretical requirement for speculative impacts. The most recent evidence in crude oil markets (Kilian and 

Murphy, 2010) also indicates a rejection of the second theoretical requirement for speculative impacts - a 

zero or near zero price elasticity of demand. In sum, the weight of the evidence at this point in time clearly 

tilts in favor of the argument that index funds did not cause a bubble in commodity futures prices.2

26. There is still a need for further research on the market impact of commodity index funds. The 

first reason is that direct tests of the relationship between index fund positions and price movements in 

energy futures markets have been hampered by the lack of publically-available data on positions of index 

funds in these markets. The second reason is ongoing concerns about the power of time-series statistical 

 

2

 Annex I of this paper contains detailed reviews of the studies cited in this section. See also Irwin and 

Sanders (2010).
TAD/CA/APM/WP(2010)8/FINAL

11

tests used in the studies that fail to find evidence of a relationship between index fund positions and 

movements in commodity futures prices. The time-series tests may lack statistical power to reject the null 

hypothesis because the dependent variable - the change in futures price - is extremely volatile. In the 

empirical analysis summarized in the following section, we attempt to address both of these deficiencies.

5. New evidence

27. Our empirical analysis relies on two related data sets compiled by the U.S. Commodity Futures 

Trading Commission (CFTC). The CFTC has long provided the breakdown of each Tuesday‘s open 

interest for U.S. markets in the Commitments of Traders (COT) report. Open interest for a given market is 

aggregated across all contract expiration months in the weekly report. The traditional COT categories 

include: commercials (hedgers), non-commercials (speculators), and non-reporting (all traders with 

position sizes below the reporting level).

28. Starting in 2007 - in response to complaints by traditional traders about the rapid increase in 

long-only index money flowing into the market - the CFTC began releasing the weekly Supplemental 

Commodity Index Traders (CIT) reports, which break out the positions of index traders for 12 agricultural 

markets. According to the CFTC, the index trader positions reflect both pension funds that would have 

previously been classified as non-commercials as well as swap dealers who would have previously been 

classified as commercials hedging OTC transactions involving commodity indices. The CIT data are 

generally considered the best glimpse of index trader activity in the 12 agricultural markets covered by the 

report. 

29. While the CIT data represent an improvement over the traditional COT data, concerns were 

expressed almost immediately that the data did not extend to other markets, particularly energy and metals 

futures. In response to requests for more information about the composition of open interest in a broader 

set of markets, the CFTC began publishing the weekly Disaggregated Commitments of Traders (DCOT) 

report in September 2009 and ultimately provided historical data back to June of 2006. The DCOT data are 

available for the same 12 agricultural markets covered by the CIT report plus a number of energy and 

metal futures markets. Like the CIT report, the positions in the DCOT report represent the combined 

futures and delta-adjusted options positions aggregated across all contracts for a particular market. 

Reporting traders are classified into four categories: swap dealers, managed money, processors and 

merchants, and other reporting traders. 

30. An important question, especially for the energy futures markets, is the degree to which the 

DCOT swap dealers category represents index fund positions. One can infer from comparisons found in the 

CFTC‘s September 2008 report on swap dealer positions (CFTC, 2008) that DCOT swap dealer positions 

in agricultural futures markets correspond reasonably closely to index trader positions. Since swap dealers 

operating in agricultural markets conduct a limited amount of non-index long or short swap transactions 

there is little error in attributing the net long position of swap dealers in these markets to index funds. 

However, swap dealers in energy futures markets conduct a substantial amount of non-index swap 

transactions on both the long and short side of the market, which creates uncertainty about how well the net 

long position of swap dealers in energy markets represent index fund positions.3For example, the CFTC 

estimates that only 41% of long swap dealer positions in crude oil futures on three dates in 2007 and 2008 

were linked to long-only index fund positions (CFTC, 2008). Despite this limitation, swap dealers are used 

in the present study as the best available proxy for index positions in the energy futures markets.

31. The CIT data are available weekly from January 3, 2006 through December 29, 2009 and the 

DCOT data are available at the same frequency starting on June 13, 2006. To facilitate the comparison of 

 

3 This was precisely the reason that the CFTC excluded energy futures markets from the CIT report. 
TAD/CA/APM/WP(2010)8/FINAL

12

the data sets and results, a common sample starting on June 13, 2006 containing 186 weekly observations 

through December 29, 2009 was used in all empirical work.

32. Index trader positions are collected for the 12 CIT agricultural markets: Chicago Board of Trade 

(CBOT) corn, CBOT soybeans, CBOT soybean oil, CBOT wheat, Kansas City Board of Trade (KCBOT) 

wheat, New York Board of Trade (NYBOT) cotton, Chicago Mercantile Exchange (CME) live cattle, 

CME feeder cattle, CME lean hogs, NYBOT coffee, NYBOT sugar, and NYBOT cocoa. Corresponding 

DCOT data are collected for these 12 CIT markets along with the DCOT data for New York Mercantile 

Exchange (NYMEX) crude oil and natural gas. The focus in the DCOT data will be on swap dealer 

positions because of their potential link to index fund positions. 

33. For the above markets, weekly futures returns (price changes) are calculated using nearby futures 

contracts, appropriately adjusting for contract roll-overs. In order to test for index trader impact on market 

variability, two measures of volatility are computed: implied volatility from the options markets and 

realized volatility as measured by Parkinson's (1980) extreme value estimator. It is important to establish 

whether or not index trader positions impact these market characteristics (returns, implied volatility, and 

realized volatility). Here, causal linkages are directly tested using Granger causality tests. 

34. A simple graphical analysis of index trader positions and market prices can be misleading. As 

shown in Figure 4 for CBOT wheat, there are periods of time - such as mid-2007 through late 2008 - where 

there appears to be a close correspondence between index trader positions and price levels. Conversely, 

there are periods, such as most of 2009, where any relationship seems remote at best. This type of 

graphical inspection is commonly presented as establishing an "obvious" link between index positions and 

prices. However, it is fraught with statistical complications and begs for a more rigorous test of the 

linkages, if any.

Figure 4. Index trader net long positions in CBOT wheat and nearby futures prices, June 2006-December 2009

$3.00

$4.00

$5.00

$6.00

$7.00

$8.00

$9.00

$10.00

$11.00

$12.00

$13.00

100

120

140

160

180

200

220

Jun-0 6

Aug-0 6

Oct-0 6

Dec-0 6

Feb-0 7

Apr-0 7

Jun-0 7

Aug-0 7

Oct-0 7

Dec-0 7

Feb-0 8

Apr-0 8

Jun-0 8

Aug-0 8

Oct-0 8

Dec-0 8

Feb-0 9

Apr-0 9

Jun-0 9

Aug-0 9

Oct-0 9

Dec-0 9

USD/Bushel

Contracts (1,000's)

Date

Index Traders Nearby Futures Prices

35. Granger causality is a standard statistical technique for determining whether one time series is 

useful in forecasting another. It is important to bear in mind that the term causality is used in a statistical 

sense, and not in a philosophical one of structural causation. More precisely a variable A is said to Granger 

cause B if knowing the time paths of B and A together improve the forecast of B based on its own time 
TAD/CA/APM/WP(2010)8/FINAL

13

path, thus providing a measure of incremental predictability. In our case the time series of interest are 

market measures of returns, implied volatility, and realized volatility, or variable B. The causal variables, 

or variable A, are measures of trader positions and speculation, including net long positions held by index 

funds, the percent of long positions held in each market by index funds, and Working's speculative index. 

36. Simply put, Granger's test asks the question: Can past values of trader positions be used to predict 

either market returns or volatility? This is a much more demanding hurdle than simply looking for a 

contemporaneous correlation or association between variables. As shown in Figure 5, there is a positive 

contemporaneous association between changes in net positions held by index traders and price changes 

(returns) in the CBOT wheat market. The simple correlation coefficient is relatively low at 0.14; but, the 

relationship is statistically significant at the 10% level. However, the magnitude of the impact is quite low 

since a 3 000 contract increase in index traders' long position is associated with just a 0.6% increase in 

prices during the same week. More importantly, this contemporaneous analysis cannot distinguish between 

the increase in index traders' positions and other correlated shifts in fundamentals: correlation does not 

imply causation. Evidence of this point is found in Figure 6, which is the same as Figure 5 except there is a 

one week time lag between the change in index fund positions and the change in the futures price. As 

clearly shown in Figure 6, increases in net index fund positions are actually followed by small (statistically 

insignificant) declines in prices the subsequent week. In this example, there is no evidence that changes in 

index traders' net long positions lead to higher (or lower) market prices.

Figure 5. Contemporaneous relationship, CBOT wheat returns (price change) and index trader net long 

positions, June 2006-December 2009

y = 0.0002x - 0.0798

R2 = 0.0187

-20%

-15%

-10%

-5 %

0 %

5 %

10%

15%

20%

-12,000 -9,000 -6,000 -3,000 0 3,000 6,000 9,000 12,000

% Change in Market Price, Week t

Change in Net Position, Week t

Note: The slope of the regression line is positive and statistically different from zero at the 10% significance level. The simple 

correlation coefficient is 0.14.
TAD/CA/APM/WP(2010)8/FINAL

14

Figure 6. Causal relationship, CBOT wheat returns (price change) and index trader net long positions, June 

2006-December 2009

y = -0.000005x - 0.1123

R2 = 0.0011

-20%

-15%

-10%

-5 %

0 %

5 %

10%

15%

20%

-12,000 -9,000 -6,000 -3,000 0 3,000 6,000 9,000 12,000

% Change in Market Price, Week t+1

Change in Net Position, Week t

Note: The slope of the regression line is negative and not statistically different from zero at the 10% significance level. The simple 

correlation coefficient is -0.03.

37. More formal Granger causality tests are conducted for a number of combinations of causal 

variables (position measures) and market characteristics. A systems approach is used to test lead-lag 

dynamics. This improves the power of statistical tests by taking into account the contemporaneous 

correlation of model residuals across markets. The system test results are summarized in Table 1. The 

formal testing failed to find any reasonably consistent causal links between trader positions and returns. 

The only statistically significant finding was a negative relationship between positions and market 

volatility. That is, there is some consistent evidence that increases in index trader positions are followed by 

lower market volatility. Even these results for market volatility must be interpreted with caution. The

possibility still exists that trader positions are correlated with some third variable that is actually causing 

market volatility to decline. 

Table 1. Causal relationships estimated for market system, June 2006 - December 2009

Causal Variable

Net Long Position

in Contracts

Percent of

Long Positions

Working's

Speculative Index

Panel A: Index Traders

 Returns No (negative) No (positive) NA

 Implied Volatility No (negative) Yes (negative) No (positive)

 Realized Volatility Yes (negative) No (positive) No (positive)

Panel B: Swap Dealers

 Returns No (positive) No (positive) NA

 Implied Volatility No (negative) Yes (negative) NA

 Realized Volatility Yes (negative) No (positive) NA

Notes: A "Yes" indicates a statistically significant (5% level) causal relationship running from the causal variables (column headings) 

to the market factors (row headings) for the overall system test. A "No" indicates that no relationship was found. The direction of the 

causal relationship is indicated by "positive" or "negative" in parenthesis, regardless of whether the impact was statistically significant 

or not.
TAD/CA/APM/WP(2010)8/FINAL

15

38. A simple statistical description of data on net positions of index traders and swap dealers is

shown in Table 2, while Table 3 provides information on the total open interest contracts held by different 

players. The main characteristics of these data can be summarized as follows:

 The overlap between index trader positions (CIT data set) and those held by swap dealers (DCOT 

data set) is quite large for the traditional grain and livestock markets. It appears to be a somewhat 

weaker correspondence for the coffee, sugar, and cocoa markets. It is clear that the swap dealer 

positions for the energy markets contain many traders other than index funds. Swap dealer 

positions are at best an imperfect proxy for index fund positions in the energy markets. 

 This is clearly seen in Table 2, which shows the net position (in contracts) held by index traders 

(Panel A) and swap dealers (Panel B) over the sample period. In Panel A, the minimum net long 

position held by index traders is never negative (short); whereas, in Panel B the minimum net 

long position for sugar, cocoa, crude oil, and natural gas is negative. In these markets, swap 

dealers clearly hold positions other than those representing long-only index investments.
TAD/CA/APM/WP(2010)8/FINAL

16

Table 2. Summary statistics, net long positions held by index traders and swap dealers (# of 

contracts) June 2006-December 2009

Panel A: Index Traders

Market Mean Maximum Minimum St. Dev.

Corn 354 043 452 568 223 985 64 877

Soybeans 140 651 198 707 89 731 26 004

Soybean Oil 66 011 77 752 36 630 10 192

CBOT Wheat 174 677 205 585 126 545 21 769

KCBOT Wheat 28 654 46 527 16 293 6 011

Cotton 84 985 122 555 57 841 15 209

Live Cattle 110 006 156 752 80 276 20 632

Feeder Cattle 7 479 10 889 4 972 1 456

Lean Hogs 80 616 127 379 46 004 18 538

Coffee 44 451 67 021 30 572 9 697

Sugar 231 756 392 740 135 745 74 836

Cocoa 18 910 31 883 5 117 5 830

Panel B: Swap Dealers

Market Mean Maximum Minimum St. Dev.

Corn 313 172 430 100 163 606 77 941

Soybeans 121 557 193 888 73 898 27 892

Soybean Oil 61 453 89 502 27 442 16 234

CBOT Wheat 142 550 189 217 91 681 25 373

KCBOT Wheat 22 073 33 863 9 952 6 906

Cotton 72 092 118 380 42 637 16 797

Live Cattle 88 844 128 967 65 368 16 351

Feeder Cattle 4 161 6 723 1 730 1 194

Lean Hogs 69 149 114 377 36 326 16 858

Coffee 37 179 56 959 21 667 8 718

Sugar 132 099 271 255 -32 149 81 371

Cocoa 8 380 16 474 -5 103 4 763

Crude Oil 40 912 106 176 -10 534 27 504

Natural Gas 49 018 253 500 -67 553 78 063

Note: Net positions are simply calculated as long positions - short positions.

 Index fund and swap dealer positions are large. In an absolute sense, the largest average

position sizes held in nearly every market is by long index funds or swap dealers. In some 

markets, such as CBOT wheat, the average position size for these traders is in excess of 

the speculative position limits. In a relative sense, index and swap dealer positions can also 

be quite large. Index traders often hold as much as 40% of the long positions in a market 

and the swap dealer category frequently holds over 30% of the long positions in a given 

market.

 Despite the large average position size, the total size of index funds within a given market 

is not overwhelming. Table 3 shows the percent of the market that is comprised of each 

trader category in the CIT (Panel A) and DCOT (Panel B) data.4In each market, the largest 

participant is a category other than index funds or swap dealers. In fact, in the CIT

categories, index traders are the smallest category in 4 of the 12 markets and the second 

smallest in the other 8 markets. The exception is swap dealers in the crude oil market who 

account for 37% of the open interest. Again, this inconsistency indicates that the link 

 

4 The denominator in these calculations is the sum of total long and short open interest, or two times either 

the long or short total open interest. 
TAD/CA/APM/WP(2010)8/FINAL

17

between swap dealer positions and index traders may be weak in the energy 

markets.

Table 3. Percent of total open interest held by CIT and DCOT categories, June 2006-December 2009

Panel A: CIT Categories

Market

NonCommercial Commercial IndexNon-Reporting

Corn 39% 35% 13% 14%

Soybeans 40% 33% 14% 14%

Soybean Oil 35% 44% 12% 8%

CBOT Wheat 41% 26% 23% 10%

KCBOT Wheat 28% 39% 12% 20%

Cotton 39% 38% 17% 6%

Live Cattle 38% 28% 20% 14%

Feeder Cattle 38% 17% 14% 31%

Lean Hogs 39% 25% 21% 14%

Coffee 43% 39% 13% 5%

Sugar 31% 44% 17% 8%

Cocoa 33% 54% 7% 6%

Panel B: DCOT Categories

Market

Managed

Money

Producers &

Merchants

Swap

Dealers

Other

Reporting

Non-Reporting

Corn 16% 32% 13% 25% 14%

Soybeans 19% 31% 13% 23% 14%

Soybean Oil 17% 42% 14% 18% 8%

CBOT Wheat 22% 23% 22% 22% 10%

KCBOT Wheat 19% 38% 10% 13% 20%

Cotton 16% 35% 18% 25% %

Live Cattle 25% 27% 18% 16% 14%

Feeder Cattle 23% 17% 9% 20% 31%

Lean Hogs 23% 25% 19% 19% 14%

Coffee 20% 37% 14% 25% 5%

Sugar 16% 39% 19% 18% 8%

Cocoa 26% 48% 12% 9% 6%

Crude Oil 18% 18% 37% 23% 3%

Natural Gas 43% 12% 28% 11% 5%

39. The empirical results of the analysis are shown in Tables 4 through 7 and can be summarised by

following general findings and representative results.

 There is no convincing evidence that positions held by index traders or swap dealers impact 

market returns. Except for a few instances in individual markets, Granger-style causality tests fail 

to reject the null hypothesis that that trader positions do not lead market returns.

 The full results for testing if CIT index traders lead market returns are shown in Table 4. In the 

individual markets, the null hypothesis of no causality can be rejected in cotton and corn at the 

5% level (with 95% confidence). This is shown by the p-values for the null hypothesis that j=0, 

j. Importantly, however, the directional impact for corn is negative while it is positive for 

cotton. This makes very little sense in the context of the current debate. Not surprisingly, the 

system-wide impact, which takes into account the opposing directional findings across markets, 

is negative (-0.4010) and indistinguishable from zero.
TAD/CA/APM/WP(2010)8/FINAL

18

Table 4. Granger causality test results for CIT net positions do not lead returns, June 2006-December 2009

   

      

m

i

n

j

Rt k k i kRt i k j k NETt j k t k

1 1

, , , , , ,    

for each market, k, and time, t.

p-value Estimate p-value

Market, k m,n j=0, j  j  j=0

Corn 1,1 0.0002 -0.1210

Soybeans 1,1 0.4206 -0.0444

Soybean Oil 1,1 0.2922 0.0874

CBOT Wheat 1,1 0.3629 0.0319

KCBOT Wheat 1,1 0.1261 -0.1460

Cotton 1,1 0.0018 0.3590

Live Cattle 2,2 0.1812 0.0008 0.9861

Feeder Cattle 2,1 0.1300 -0.3730

Lean Hogs 1,1 0.2078 -0.1320

Coffee 1,1 0.3348 -0.1730

Sugar 1,1 0.2647 -0.0520

Cocoa 1,1 0.4591 0.1610

p-value Estimate p-value

j,k=0, j,k  j,k  j,k=0

System 0.0001 -0.4010 0.3836

Note:  j values are taken to the 105power. 

Technical Note: The models are estimated across the K markets as an SUR system. Wald tests could not reject the following crossmarket coefficient restrictions: α1= α2=...= αK ; γ1,1= γ1,2=...=γ1,K; and γ2,1= γ2,2 =...=γ2,K for all K markets. These restrictions are imposed 

on the system and the common coefficients are estimated as a single pooled parameter across all K markets. 

 Larger long positions by index traders and swap dealers lead to lower market volatility in a Granger 

sense. There is a consistent tendency across a number of position and volatility measures to reject 

the null hypothesis that index trader positions do not lead market volatility. The direction of the 

impact is routinely negative. While index positions lead to lower volatility in a statistical sense, it 

is possible that trader positions coincide with some other fundamental variable that is actually 

causing the lower market volatility. Still, this result is contrary to popular notions about index 

traders increasing market volatility.

 These general conclusions apply to both the volatility implied in the options markets and realized 

volatility. As a representative example, consider the Granger causality test of the null hypothesis 

that DCOT swap dealers' net positions do not lead realised market volatility. The system estimation

results are presented in Table 5. The null hypothesis is rejected at the 5% level in soybeans and 

cocoa. In both of these markets, the directional impact is negative: increases in net long positions 

held by swap dealers predict lower market volatility in the subsequent week. More convincing than 

the individual market results, the system results show that the aggregate directional impact is 

statistically negative (-36.1) with nearly 99% confidence (1 - 0.0131).
TAD/CA/APM/WP(2010)8/FINAL

19

Table 5. Granger causality test results for DCOT swap dealer net positions do not lead realized volatility, June 

2006-December 2009

   

      

m

i

n

j

RVt k k i kRVt i k j k NETt j k t k

1 1

, , , , , ,    

for each market, k, and time, t.

p-value Estimate p-value

Market m,n j=0, j  j  j=0

Corn 2,1 0.8258 0.2000

Soybeans 4,1 0.0242 -3.3700

Soybean Oil 2,1 0.5347 -0.9500

CBOT Wheat 2,1 0.6975 0.4370

KCBOT Wheat 3,1 0.1308 -5.5000

Cotton 3,1 0.9358 0.2340

Live Cattle 3,1 0.0600 -2.4600

Feeder Cattle 3,1 0.5317 -5.8200

Lean Hogs 3,1 0.1531 3.7900

Coffee 1,2 0.1568 -11.8200 0.0581

Sugar 3,1 0.8018 -0.3200

Cocoa 4,1 0.0420 -12.0300

Crude Oil 3,1 0.0889 1.0500

Natural Gas 1,1 0.5975 0.4610

p-value Estimate p-value

j,k=0, j,k  j,k  j,k=0

System 0.0408 -36.1000 0.0131

Note:  j values are taken to the 105power. 

Technical Note: The models are estimated across the K markets as an SUR system. Wald tests could not reject the following crossmarket coefficient restrictions: γ3,1= γ3,2 =...=γ23,K for all K markets. These restrictions are imposed on the system and the common 

coefficients are estimated as a single pooled parameter across all K markets. 

 Excessive speculation - as measured by Working's T-index - is associated with greater 

subsequent variability in a few markets. These results conflict with negative relationships found 

between index trader positions and market volatility. The contrasting results suggests that 

excessive speculation is broader than just index fund activity and may be better measured with 

Working's T-index, which measures excessive speculation relative to hedging demands. 

 Table 6 shows the summary statistics for Working's T-index adjusted for index trader positions. 

For example, the average T-index for corn is 1.15 - indicating speculation in the corn market is 

15% greater than that needed to meet hedging needs. Historically, this would have been 

considered a potentially inadequate amount of speculation to efficiently meet hedging demands

and facilitate the transfer of risk. Notably, some of the markets with high T-values (livestock and 

CBOT wheat) are also those markets with a relatively high portion of index traders (see Table 3, 

Panel A). Still, even in these markets, the maximums are not beyond those recorded by prior 

researchers, the average values are near historic norms, and the minimums could be considered 

inadequate.
TAD/CA/APM/WP(2010)8/FINAL

20

Table 6. Summary statistics, working's speculative T-Index, adjusted for index trader positions, June 2006-

December 2009

Market Mean Maximum Minimum St. Dev.

Corn 1.15 1.34 1.07 0.06

Soybeans 1.17 1.53 1.09 0.09

Soybean Oil 1.12 1.36 1.04 0.07

CBOT Wheat 1.44 1.87 1.19 0.16

KCBOT Wheat 1.18 1.34 1.08 0.06

Cotton 1.16 1.48 1.03 0.11

Live Cattle 1.33 1.50 1.15 0.07

Feeder Cattle 1.86 3.28 1.32 0.38

Lean Hogs 1.43 2.01 1.17 0.19

Coffee 1.17 1.41 1.04 0.08

Sugar 1.15 1.26 1.06 0.04

Cocoa 1.14 1.28 1.06 0.05

Technical Note: Working’s speculative “T” index is easily calculated using the traditional COT trader categories:

T = 1+ SS / (HL + HS) if (HS ≥ HL)

or

T = 1 + SL / (HL + HS) if (HL > HS)

where open interest held by speculators (non-commercials) and hedgers (commercials) is denoted as follows: SS = Speculation, 

Short; SL = Speculation, Long; HL = Hedging, Long; and HS = Hedging, Short. 

 Working's T-index is silent on the direction of speculation (long versus short). Instead, the 

amount of speculation is gauged relative to what is needed to balance hedging positions. Because 

it is directionless Working's T-index is only tested as a causal variable for market volatility. 

Table 7 shows the results for testing if the T-index Granger causes realized market volatility. 

Granger causality is found in 4 markets at the 95% confidence level. In all 4 markets, the 

directional impact is positive - higher levels of excessive speculation as measured by Working's T 

are followed by greater realized market volatility. For example, if the speculative index in lean 

hogs increases by 0.10, then actual volatility the following week increases by 1.18%. These 

individual market results are notable in comparison to the negative directional impacts found 

when simply measuring speculation with net index fund positions (Table 5). Still, the impact is 

not pervasive across markets as no system impact is found at even a modest confidence level. 
TAD/CA/APM/WP(2010)8/FINAL

21

Table 7. Granger causality test results for T-Index does not lead realized volatility, June 2006-December 2009

   

     

m

i

n

j

RVt k k i kRVt i k j kTIndext j k t k

1 1

, , , , , ,    

for each market, k, and time, t.

p-value Estimate p-value

Market m,n j=0, j  j  j=0

Corn 1,1 0.0470 24.8261

Soybeans 4,1 0.6982 -2.5196

Soybean Oil 2,1 0.7590 2.3205

CBOT Wheat 2,1 0.5745 -1.7284

KCBOT Wheat 3,1 0.7993 -1.8937

Cotton 3,1 0.4823 -4.7687

Live Cattle 3,1 0.3602 3.2854

Feeder Cattle 3,1 0.0208 1.8090

Lean Hogs 3,1 0.0003 11.7991

Coffee 1,1 0.6234 -4.0321

Sugar 4,1 0.2101 -30.5000

Cocoa 4,1 0.0308 34.0968

p-value Estimate p-value

j,k=0, j,k  j,k  j,k=0

System 0.0028 32.6945 0.3844

Technical Note: The models are estimated across the K markets as an SUR system. Wald tests could not reject the following crossmarket coefficient restrictions: γ2,1= γ2,2 =...=γ2,K ; γ3,1= γ3,2 =...=γ3,K for all K markets. These restrictions are imposed on the system 

and the common coefficients are estimated as a single pooled parameter across all K markets. 

40. In sum, our results tilt the weight of the evidence even further in favour of the argument that 

index funds did not cause a bubble in commodity futures prices.5The evidence in our study is strongest for 

the agricultural futures markets because the data on index trader positions are measured with reasonable 

accuracy. The evidence is not as strong in the two energy markets studied because of considerable 

uncertainty about the degree to which the available data actually reflect index trader positions in these 

markets. Perhaps the most surprising result is the consistent tendency for increasing index fund positions to 

be associated with declining volatility. Caution must be exercised in interpreting this finding as a third 

factor common to all markets may be in fact be generating the decline in volatility. Nonetheless, this result 

is contrary to popular notions about the market impact of index funds, but is not so surprising in light of the 

traditional problem in commodity futures markets of the inadequacy of speculation (see Sanders, Irwin, 

and Merrin, 2010). These results imply that more research in this area is needed to understand the present 

role of speculation in futures markets.

6. Policy Conclusions

41. The empirical evidence presented in this preliminary study does not appear at present to warrant 

extensive changes in the regulation of index funds participation in agricultural commodity markets; any 

such changes require careful consideration so as to avoid unintended negative impacts. For example,

limiting the participation of index fund investors could unintentionally deprive commodity futures markets 

of an important source of liquidity and risk-absorption capacity at times when both are in high demand.

 

5 Annex I of this paper contains a detailed presentation of all statistical test results. See also Irwin and 

Sanders (2010).
TAD/CA/APM/WP(2010)8/FINAL

22

This could make commodity futures markets less efficient mechanisms for transferring risk from parties 

who do not want to bear it to those that do, creating added costs that ultimately are passed back to 

producers in the form of lower prices and to consumers as higher prices. 

42. These conclusions do not imply that commodity futures markets have functioned flawlessly

during the last several years. In particular, the lack of consistently acceptable convergence performance for 

CBOT corn, soybean, and wheat contracts since late 2005 has been widely discussed (e.g., Henriques, 

2008). The failure of cash and futures prices to convergence at contract expiration has existed for extended 

and varied periods. Performance has been consistently weakest in wheat, with delivery location basis at 

times exceeding one dollar per bushel, a level of disconnect between cash and futures not previously 

experienced in grain futures markets. The possible role of index funds in contributing to convergence 

problems has also been widely discussed (USS/PSI, 2009). Further research is needed to better understand 

the impact of index fund trading on this aspect of commodity market performance as well as the 

fundamental role of speculation in these markets.
TAD/CA/APM/WP(2010)8/FINAL

23

REFERENCES

Aulerich, N.M., S.H. Irwin, and P. Garcia, (2010), ―The Price Impact of Index Funds in Commodity 

Futures Markets: Evidence from the CFTC‘s Daily Large Trader Reporting System‖. Working 

Paper, Department of Agricultural and Consumer Economics, University of Illinois at UrbanaChampaign. 

Buyuksahin, B., and J.H. Harris, (2009), ―The Role of Speculators in the Crude Oil Futures Markets.‖ 

Working Paper, U.S. Commodity Futures Trading Commission. 

Commodity Futures Trading Commission (CFTC) (2008), ―Staff Report on Commodity Swap Dealers & 

Index traders with Commission Recommendations‖.

Davis, A., (2009), ―Cargill‘s Inside View Helps It Buck Downturn.‖ The Wall Street Journal, January 14. 

Einloth, J., (2009), ―Speculation and Recent Volatility in the Price of Oil.‖ Working Paper, Division of 

Insurance and Research, Federal Deposit Insurance Corporation. 

Gilbert, C.L., (2009), ―Speculative Influences on Commodity Futures Prices, 2006-2008.‖ Working Paper, 

Department of Economics, University of Trento. 

Hamilton, J.D., (2009), ―Causes and Consequences of the Oil Shock of 2007-08.‖ Brookings Papers on 

Economic Activity:215-261.

Headey, D., and S. Fan, (2008), ―Anatomy of a Crisis: The Causes and Consequences of Surging Food 

Prices.‖ Agricultural Economics 39:375-391.

Irwin, S.H., and D.R. Sanders, (2010), ―The Impact of  ̳Financialization‘ in Commodity Futures Markets.‖ 

Technical Report, Organisation on Economic Co-Operation and Development.

Irwin, S.H., D.R. Sanders, and R.P. Merrin, (2009), ―Devil or Angel? The Role of Speculation in the 

Recent Commodity Price Boom (and Bust).‖ Journal of Agricultural and Applied Economics

41:393-402.

Kilian, L., and D. Murphy, (2010), ―The Role of Inventories and Speculative Trading in the Global Market 

for Crude Oil.‖ Working Paper, Department of Economics, University of Michigan. 

Krugman, P., (2008), ―More on Oil and Speculation.‖ New York Times, May 13. 

Masters, M.W., (2008), ―Testimony before the Committee on Homeland Security and Government Affairs, 

U.S. Senate.‖ May 20. 

Masters, M.W. and A.K. White ,(2008), ―The Accidental Hunt Brothers: How Institutional Investors are 

Driving up Food and Energy Prices.‖
TAD/CA/APM/WP(2010)8/FINAL

24

Parkinson, M. (1980), ―The Extreme Value Method for Estimating the Variance of the Rate of 

Return.‖ Journal of Business 53:61–65.

Petzel, T. (2009), ―Testimony before the CFTC.‖ July 28. 

Sanders, D.R., and S.H. Irwin, (2010a), ―A Speculative Bubble in Commodity Futures Prices? 

Cross-Sectional Evidence.‖ Agricultural Economics 41:25-32.

Sanders, D.R., and S.H. Irwin, (2010b), ―Bubbles, Froth, and Facts: The Impact of Index Funds on 

Commodity Futures Prices.‖ Working Paper, Department of Agricultural and Consumer Economics, 

University of Illinois at Urbana-Champaign,

Sanders, D.R., S.H. Irwin, and R.P. Merrin, (2010), ―The Adequacy of Speculation in Agricultural 

Futures Markets: Too Much of a Good Thing?‖ Applied Economics Perspectives and Policy

32:77-94

Smith, J.L., (2009), ―World Oil: Market or Mayhem? Journal of Economic Perspectives 23:145-164.

Stoll, H.R., and R.E.Whaley, (2009), “Commodity Index Investing and Commodity Futures Prices”,

Working Paper, Owen Graduate School of Management, Vanderbilt University.

Tang, K., and W. Xiong, (2010), ―Index Investing and the Financialization of Commodities”. Working 

Paper, Department of Economics, Princeton University.

Till, H., (2009), ―Has There Been Excessive Speculation in the US Oil Futures Markets?‖ EDHEC-Risk 

Institute Position Paper.

United States Senate, Permanent Subcommittee on Investigations (USS/PSI), (2009), ―Excessive 

Speculation in the Wheat Market‖

Working, H. ―Speculation on Hedging Markets, (1960),‖ Food Research Institute Studies 1:185-220.

Wright, B., (2009), ―International Grain Reserves and Other Instruments to Address Volatility in Grain 

Markets.‖ Policy Research Working Paper 5028, The World Bank.
TAD/CA/APM/WP(2010)8/FINAL

25

GLOSSARY OF TERMS6

Arbitrage: A strategy involving the simultaneous purchase and sale of identical or equivalent commodity 

futures contracts or other instruments across two or more markets in order to benefit from a 

discrepancy in their price relationship. In a theoretical efficient market, there is a lack of opportunity 

for profitable arbitrage. 

Back Months: Futures delivery months other than the spot or front month (also called deferred months). 

Bear: One who expects a decline in prices. The opposite of a bull. A news item is considered bearish if it 

is expected to result in lower prices. 

Bear Market: A market in which prices generally are declining over a period of months or years. The 

opposite of bull market. 

Board of Trade: Any organized exchange or other trading facility for the trading of futures and/or option 

contracts. 

Bull: One who expects a rise in prices. The opposite of bear. A news item is considered bullish if it is 

expected to result in higher prices. 

Bull Market: A market in which prices generally are rising over a period of months or years. Opposite of 

bear market. 

Buyer: A market participant who takes a long futures position or buys an option. An option buyer is also 

called a taker, holder, or owner. 

Cash Commodity: The physical or actual commodity as distinguished from the futures contract, 

sometimes called spot commodity or actuals. 

Cash Price: The price in the marketplace for actual cash or spot commodities to be delivered via 

customary market channels. 

CFTC Form 40: The form used by large traders to report their futures and option positions and the 

purposes of those positions. 

Closing Price: The price recorded during trading that takes place in the final period of a trading session‘s 

activity that is officially designated as the "close." 

Commercial: An entity involved in the production, processing, or merchandising of a commodity. 

 

6 The terms and definitions used in the glossary are taken primarily from the Commodity Futures Trading 

Commission's "A Guide to the Language of the Futures Industry" which can be accessed online at 

http://www.cftc.gov/ConsumerProtection/EducationCenter/CFTCGlossary/index.htm.
TAD/CA/APM/WP(2010)8/FINAL

26

Commitments of Traders Report (COT): A weekly report from the CFTC providing a breakdown of 

each Tuesday's open interest for markets in which 20 or more traders hold positions equal to or 

above the reporting levels established by the CFTC. Open interest is broken down by aggregate 

commercial, non-commercial, and non-reportable holdings. 

Commodity Futures Trading Commission (CFTC): The Federal regulatory agency established by the 

Commodity Futures Trading Act of 1974 to administer the Commodity Exchange Act. 

Commodity Index: An index of a specified set of (physical) commodity prices or commodity futures 

prices. 

Commodity Index Fund: An investment fund that enters into futures or commodity swap positions for the 

purpose of replicating the return of an index of commodity prices or commodity futures prices. 

Commodity Index Swap: A swap whose cash flows are intended to replicate a commodity index. 

Commodity Index Trader: An entity that conducts futures trades on behalf of a commodity index fund or 

to hedge commodity index swap positions. 

Commodity-Linked Bond: A bond in which payment to the investor is dependent to a certain extent on 

the price level of a commodity, such as crude oil, gold, or silver, at maturity. 

Commodity Pool: An investment trust, syndicate, or similar form of enterprise operated for the purpose of 

trading commodity futures or option contracts. Typically thought of as an enterprise engaged in the 

business of investing the collective or ―pooled‖ funds of multiple participants in trading commodity 

futures or options, where participants share in profits and losses on a pro rata basis. 

Commodity Pool Operator (CPO): A person engaged in a business similar to an investment trust or a 

syndicate and who solicits or accepts funds, securities, or property for the purpose of trading 

commodity futures contracts or commodity options. The commodity pool operator either itself 

makes trading decisions on behalf of the pool or engages a commodity trading advisor to do so. 

Commodity Trading Advisor (CTA): A person who, for pay, regularly engages in the business of 

advising others as to the value of commodity futures or options or the advisability of trading in 

commodity futures or options, or issues analyses or reports concerning commodity futures or 

options. 

Commodity Swap: A swap in which the payout to at least one counterparty is based on the price of a 

commodity or the level of a commodity index. 

Corner: (1) Securing such relative control of a commodity that its price can be manipulated, that is, can be 

controlled by the creator of the corner; or (2) in the extreme situation, obtaining contracts requiring 

the delivery of more commodities than are available for delivery. 

Counterparty: The opposite party in a bilateral agreement, contract, or transaction, such as a swap. 

Delivery: The tender and receipt of the actual commodity, the cash value of the commodity, or of a 

delivery instrument covering the commodity (e.g., warehouse receipts or shipping 

Disaggregated Commitments of Traders Report (DCOT): A weekly report from the CFTC providing a 

breakdown of each Tuesday's open interest for markets in which 20 or more traders hold positions 

equal to or above the reporting levels established by the CFTC. Open interest is broken down by 
TAD/CA/APM/WP(2010)8/FINAL

27

managed money, swap dealers, producers and merchants, other reporting traders, and non-reporting 

traders.

Efficient Market: In economic theory, an efficient market is one in which market prices adjust rapidly to 

reflect new information. The degree to which the market is efficient depends on the quality of 

information reflected in market prices. In an efficient market, profitable arbitrage opportunities do 

not exist and traders cannot expect to consistently outperform the market unless they have lowercost access to information that is reflected in market prices or unless they have access to information 

before it is reflected in market prices. 

Exchange Traded Fund (ETF): An investment vehicle holding a commodity or other asset that issues 

shares that are traded like a stock on a securities exchange. 

Front Month: The spot or nearby delivery month, the nearest traded contract month. 

Fund of Funds: A commodity pool that invests in other commodity pools rather than directly in futures 

and options contracts. 

Futures Commission Merchant (FCM): Individuals, associations, partnerships, corporations, and trusts 

that solicit or accept orders for the purchase or sale of any commodity for future delivery on or 

subject to the rules of any exchange and that accept payment from or extend credit to those whose 

orders are accepted. 

Futures Contract: An agreement to purchase or sell a commodity for delivery in the future: (1) at a price 

that is determined at initiation of the contract; (2) that obligates each party to the contract to fulfill 

the contract at the specified price; (3) that is used to assume or shift price risk; and (4) that may be 

satisfied by delivery or offset. 

Futures-equivalent: A term frequently used with reference to speculative position limits for options on 

futures contracts. The futures-equivalent of an option position is the number of options multiplied by 

the previous day's risk factor or delta for the option series. For example, ten deep out-of-money 

options with a delta of 0.20 would be considered two futures-equivalent contracts. The delta or risk 

factor used for this purpose is the same as that used in delta-based margining and risk analysis 

systems. 

Futures Option: An option on a futures contract. 

Futures Price: (1) Commonly held to mean the price of a commodity for future delivery that is traded on a 

futures exchange; (2) the price of any futures contract. 

Hedge Exemption: An exemption from speculative position limits for bona fide hedgers and certain other 

persons who meet the requirements of exchange and CFTC rules. 

Hedge Fund: A private investment fund or pool that trades and invests in various assets such as securities, 

commodities, currency, and derivatives on behalf of its clients, typically wealthy individuals. Some 

commodity pool operators operate hedge funds. 

Hedger: A trader who enters into positions in a futures market opposite to positions held in the cash 

market to minimize the risk of financial loss from an adverse price change; or who purchases or sells 

futures as a temporary substitute for a cash transaction that will occur later. One can hedge either a 

long cash market position (e.g., one owns the cash commodity) or a short cash market position (e.g., 

one plans on buying the cash commodity in the future). 
TAD/CA/APM/WP(2010)8/FINAL

28

Historical Volatility: A statistical measure (specifically, the annualized standard deviation) of the 

volatility of a futures contract, security, or other instrument over a specified number of past trading 

days. 

Implied Volatility: The volatility of a futures contract, security, or other instrument as implied by the 

prices of an option on that instrument, calculated using an option pricing model. 

Large Traders: A large trader is one who holds or controls a position in any one future or in any one 

option expiration series of a commodity on any one exchange equaling or exceeding the exchange or 

CFTC-specified reporting level. 

Long: (1) One who has bought a futures contract to establish a market position; (2) a market position that 

obligates the holder to take delivery; (3) one who owns an inventory of commodities. 

Long Hedge: Hedging transaction in which futures contracts are bought to protect against possible 

increases in the cost of commodities. 

Managed Money Traders (MMTs): Futures market participants who engage in futures trades on behalf 

of investment funds or clients. While MMTs are commonly equated with hedge funds, they may 

include Commodity Pool Operators and other managed accounts as well as hedge funds. While 

CFTC Form 40 does not provide a place to declare oneself a Managed Money Trader, a large trader 

can declare itself a ―Hedge Fund (H)‖ or ―Managed Accounts and Commodity Pools.‖ 

Manipulation: Any planned operation, transaction, or practice that causes or maintains an artificial price. 

Specific types include corners and squeezes as well as unusually large purchases or sales of a 

commodity or security in a short period of time in order to distort prices, and putting out false 

information in order to distort prices. 

Nearby Delivery Month: The month of the futures contract closest to maturity; the front month or lead 

month. 

Offset: Liquidating a purchase of futures contracts through the sale of an equal number of contracts of the 

same delivery month, or liquidating a short sale of futures through the purchase of an equal number 

of contracts of the same delivery month. 

Open Interest: The total number of futures contracts long or short in a delivery month or market that has 

been entered into and not yet liquidated by an offsetting transaction or fulfilled by delivery. 

Option: A contract that gives the buyer the right, but not the obligation, to buy or sell a specified quantity 

of a commodity or other instrument at a specific price within a specified period of time, regardless of 

the market price of that instrument. Also see Put and Call. 

Over-the-Counter (OTC): The trading of commodities, contracts, or other instruments not listed on any 

exchange. OTC transactions can occur electronically or over the telephone. Also referred to as OffExchange. 

Physical Delivery: A provision in a futures contract or other derivative for delivery of the actual 

commodity to satisfy the contract. 

Position: An interest in the market, either long or short, in the form of one or more open contracts. 
TAD/CA/APM/WP(2010)8/FINAL

29

Price Discovery: The process of determining the price level for a commodity based on supply and demand 

conditions. Price discovery may occur in a futures market or cash market. 

Reporting Level: Sizes of positions set by the exchanges and/or the CFTC at or above which commodity 

traders or brokers who carry these accounts must make daily reports about the size of the position by 

commodity, by delivery month, and whether the position is controlled by a commercial or noncommercial trader. 

Rolling Futures Positions: The lifting a near futures position and re-establishing it in a more deferred 

delivery month. 

Short: (1) The selling side of an open futures contract; (2) a trader whose net position in the futures market 

shows an excess of open sales over open purchases. See Long. 

Short Hedge: Selling futures contracts to protect against possible decreased prices of commodities. 

Small Traders: Traders who hold or control positions in futures or options that are below the reporting 

level specified by the exchange or the CFTC. 

Speculative Bubble: A rapid run-up in prices caused by excessive buying that is unrelated to any of the 

basic, underlying factors affecting the supply or demand for a commodity or other asset. Speculative 

bubbles are usually associated with a "bandwagon" effect in which speculators rush to buy the 

commodity (in the case of futures, "to take positions") before the price trend ends, and an even 

greater rush to sell the commodity (unwind positions) when prices reverse. 

Speculative Position Limit: The maximum position, either net long or net short, in one commodity future 

(or option) or in all futures (or options) of one commodity combined that may be held or controlled 

by one person (other than a person eligible for a hedge exemption) as prescribed by an exchange 

and/or by the CFTC. 

Speculator: In commodity futures, a trader who does not hedge, but who trades with the objective of 

achieving profits through the successful anticipation of price movements. 

Spread: The purchase of one futures delivery month against the sale of another futures delivery month of 

the same commodity; the purchase of one delivery month of one commodity against the sale of that 

same delivery month of a different commodity; or the purchase of one commodity in one market 

against the sale of the commodity in another market, to take advantage of a profit from a change in 

price relationships. The term spread is also used to refer to the difference between the price of a 

futures month and the price of another month of the same commodity. A spread can also apply to 

options. 

Squeeze: A market situation in which the lack of supplies tends to force shorts to cover their positions by 

offset at higher prices. 

Supplemental Commodity Index Traders (CIT): A weekly report from the CFTC providing a 

breakdown of each Tuesday's open interest for markets in which 20 or more traders hold positions 

equal to or above the reporting levels established by the CFTC. Open interest is broken down by 

commercial, non-commercial, index traders, and non-reportable holdings.

Swap: In general, the exchange of one asset or liability for a similar asset or liability for the purpose of 

lengthening or shortening maturities, or otherwise shifting risks. This may entail selling one 

securities issue and buying another in foreign currency; it may entail buying a currency on the spot 
TAD/CA/APM/WP(2010)8/FINAL

30

market and simultaneously selling it forward. Swaps also may involve exchanging income flows; for 

example, exchanging the fixed rate coupon stream of a bond for a variable rate payment stream, or 

vice versa, while not swapping the principal component of the bond. Swaps are generally traded 

over-the-counter. 

Swap Dealer (AS): An entity such as a bank or investment bank that markets swaps to end users. Swap 

dealers often hedge their swap positions in futures markets. Alternatively, an entity that declares 

itself a ―Swap/Derivatives Dealer‖ on CFTC Form 40.

Underlying Commodity: The cash commodity underlying a futures contract. Also, the commodity or 

futures contract on which a commodity option is based, and which must be accepted or delivered if 

the option is exercised. 

Volatility: A statistical measurement (the annualized standard deviation of returns) of the rate of price 

change of a futures contract, security, or other instrument underlying an option. See Historical 

Volatility, Implied Volatility. 

Volume: The number of contracts traded during a specified period of time. It is most commonly quoted as 

the number of contracts traded, but for some physical commodities may be quoted as the total of 

physical units, such as bales, bushels, or barrels.