WTI Crude


Brent Crude


Natural Gas




Heating Oil


Rotate device for more commodity prices

Alt Text

The Mumbo-Jumbo of the Oil Markets

the systems that operate in…

Alt Text

Iran's Patience May Pay Off in Oil Markets

On Thursday, Iranian President Hassan…

Alt Text

China’s Stock Market Meltdown Not Over Yet

While many felt that the…

Index Funds and Commodity Prices

There has been a lot of growth over the last decade in funds that take long positions in commodity futures contracts in order to offer investors an asset that follows raw commodity prices. I've been looking into some of the data that have been used to measure the size of those positions.

Hedge fund manager Michael Masters received a lot of attention in 2008 with his testimony before the U.S. Senate on the role that "financialization" of commodity futures markets may have played in the big increases in commodity prices being seen at the time. One of the ways he made his point was with a graph comparing the price of oil with an estimate of the number of oil futures contracts held by index funds who were buying these contracts on behalf of investors seeking direct exposure to commodity prices. Stanford Professor Ken Singleton has recently updated a graph illustrating the correlation, in which the black line is the price of crude oil and the dashed is an estimate of futures contracts held by index funds.

Oil Futures contracts
Solid line: Price of West Texas Intermediate using near-month NYMEX futures price in dollars per barrel (left scale). Dashed line: Long positions in oil futures contracts of commodity index funds as inferred using the methodology of Masters (2008) (in numbers of contracts or equivalently thousands of barrels, right scale). Source: Singleton (2011).

How does one come up with an estimate of the number of futures contracts held by index funds? Masters relied on the Supplemental Commitments of Traders (SCOT) that the Commodity Futures Trading Commission began reporting in 2007. The CFTC used forms and interviews to identify traders in 12 agricultural commodities who were "replicating a commodity index by establishing long futures positions in the component markets and then rolling those positions forward from future to future using a fixed methodology". Of these 12 commodities, one (soybean oil) is included in the Dow Jones-AIG commodity index but not in the Goldman Sachs Commodity Index. Masters' assumption was that all of the index fund positions in soybean oil in the SCOT report were also holding crude oil futures in the proportions that somebody trying to replicate the DJ-AIG would have to hold. The SCOT also includes two agricultural commodities (KC wheat and feeder cattle) that are included in the Goldman Sachs Commodity Index (GSCI) but not in the DJ-AIG. Masters assumed that one or the other of these were held solely by funds seeking to replicate the GSCI, from which one gets an imputed volume of crude oil contracts held by those funds. Adding those two numbers of imputed oil futures holdings produces the dashed line in the figure above.

One immediate check on the validity of this procedure is whether the number of GSCI crude oil contracts implied by KC wheat estimates is similar to the number of GSCI crude oil contracts implied by feeder cattle. In the example used in the appendix to Masters (2008), which is based on January 17, 2006 numbers, the estimates are indeed similar. However, Irwin and Sanders (2011) note that for other dates the discrepancies are large. For example, Masters' methodology would infer from the December 31, 2007 SCOT report that GSCI-based index funds were holding 286,000 crude oil contracts if the estimate was based on feeder cattle positions but 468,000 contracts if based on KC wheat positions. Also, as noted by Irwin and Sanders (2011) and Singleton (2011), a small deviation from assumptions results in a huge error in the inference. In particular, feeder cattle only have a weight of 0.47% in the GSCI, which means that number gets multiplied by 213 to produce an estimate of crude oil holdings. Any error in the assumptions is thus getting multiplied by several hundred before generating an inferred value for crude oil. An additional problem is that the reported positions of index traders include short positions, something a pure index fund would not do, and again inconsistent with the methodology.

Given the need for better data, since 2008 the CFTC has been conducting an on-going "special call" of all swap dealers and index funds known to be significant users of U.S. futures markets, which it summarizes in a separate Index Investment Data report. Irwin and Sanders (2011) describe the source of these data:

Each firm's entire book related to commodity index investments is reported, not just the netted amount that may be managed ultimately in the futures markets. The Index Investment Data (IID) is cross-checked by comparing it to the positions reported through the CFTC's larger trader reporting system and by engaging the firms in extensive discussions.

Another huge advantage of the IID data is that it includes direct estimates of crude oil futures contracts, rather than trying to impute them from holdings such as soybean oil.

Irwin and Sanders used the Masters methodology to infer positions of crude oil positions of index fund traders as of the end of each quarter, which is summarized by the dashed line in the figure below. For comparison, the long positions in crude oil contracts of index-fund traders as reported in the IID data is shown in the solid line. This comparison suggests that the Masters methodology is likely off by huge amounts.

Long positions in oil futures contracts
Dashed line: Long positions in oil futures contracts of commodity index funds as inferred using the methodology of Masters (2008) (in thousands of contracts or equivalently millions of barrels), quarterly 2007:Q4-2011:Q1. Solid line: long positions in oil futures contracts of commodity index funds as inferred from the CFTC Investment Index Data. Source: Irwin and Sanders (2011).

Unfortunately, the IID data are not available at higher frequencies, so some researchers are still making use of Masters-type imputations. Singleton's (2011) regressions suggest that the imputed series can help predict crude oil futures prices. Given the evidence above, this result seems puzzling.

By. James Hamilton

Reproduced from Econbrowser

Back to homepage

Leave a comment
  • Anonymous on August 09 2011 said:
    By using the Index Investment Data report as the yardstick by which tomeasure the accuracy of the Masters methodology, Mr. Hamilton has put the cart before the horse. He misses the critical point that many index fund providers hedge with physicals as well as long futures and that theMasters data refers to barrel-equivalent holdings, which includes bothfutures and physicals. Physical holdings by swap dealers do not show up in the monthly Index Investment Data report, yet it has been widely reported that swap dealers and other index fund providers have been buying physical warehousing and other storage capacity for energy and metals (e.g.http://online.wsj.com/article/SB10001424052702304803104576426131256469252.html).
  • Anonymous on August 09 2011 said:
    (continued) Masters' methodology is therefore the best approach available forestimating total crude oil holdings by index traders. The fact thatacademics on both sides of the debate have embraced it is a clearendorsement of its validity (e.g. Pirrong 2008, Singleton 2011). The CIT report data, which Mr. Hamilton calls "SCOT" data, remains thebest publicly available data for time series analysis of index fundactivity. The Index Investment Data report is simply not frequent enoughto be useful.
  • Anonymous on August 10 2011 said:
    Hedge with long futures. What is that all about?In one of the first lectures I gave my finance students I made it clear that - for the most part - there is not enough liqidity in long futures for efficient hedging to take place. Something else: if you think that the word of Mr Pirrong is worth something in a discussion involving oil, you have sadly missed the point.

Leave a comment

Oilprice - The No. 1 Source for Oil & Energy News