Commodity Prices and Data Sources

August 17, 2010

I’ve spent days or weeks looking for good online sources of commodity prices and related economic data.

For large volumes of data, suggest a subscription to Thompson/Datastream or a Bloomberg Terminal.  Be prepared to pay $5000 or so per year per terminal.

For Commodities Tick Data, I have used and recommend TickData.com.  You not only get the data, but also a frequency conversion tool.

This compilation is Copyright ©2010-2012 William Smith, commoditymodels.com

Many Commodities – Monthly

Agriculturals

Prices

  • US Historical Land Values (1997 onwards) USDA
  • US Historical Crop Prices USDA

Data

Metals

Prices

  • Copper Futures Prices on CME
  • Gold Futures Prices on CME

Data

Energy

Prices

Data

Exotics

Economics

If the data or price you want is not here, please don’t ask me to find it for you (unless you want to pay me for consulting), find it yourself and then post a comment and I’ll update the table.

This compilation is Copyright ©2010 William Smith, commoditymodels.com


Bloomberg’s Commodity “Plunge” Misreporting

June 3, 2010

Avid viewers and readers of Bloomberg, like myself, may have noticed a recent article “Commodities’ Biggest Drop Since Lehman Is A Bear Signal”. They use a little-quoted index of commodities, the “Journal of Commerce Industrial Price Commodity Smoothed Price Index”, claim that it “reflects clearer signs of supply on demand because half the items it tracks don’t trade on exchanges used by speculators” and cite a 57% “plunge” in May!

For me, as a commodity researcher, this seems like huge news. Bloomberg seems to be saying that real-world, unmanipulated commodity prices have halved in a month, and speculator’s presence in the bigger markets is hiding this ‘reality’ from investors. Right?

Absolutely wrong. Dangerous misreporting, Bloomberg. I now see people picking up this story around the world on their blogs and suggesting this may be a time to sell investments.

The Bloomberg TV report (I’m trying to get a screen capture, but can’t find the video archive) even compounds this error by showing this -57% figure underneath daily and monthly prices changes for other assets such as WTI crude oil.

First lesson, go to the source and understand your data. Bloomberg, quote the “Journal of Commerce Industrial Price Commodity Smoothed Price Index” as being at 60.56 at the end of April 2010, and 25.97 at the end of May 2010. That’s a drop of 57%, as they state.

So, what is this index measuring? And, how can a commodity price index be negative, as it was in July 2009? The main problem, it’s not a price index! It measures the year-on-year growth, as a percentage, of the tracked commodity prices over the previous year. Confusingly, the Bloomberg page states “the base rate of this index is 2006=100”. To me, this bit is plain wrong. A quick bit of charting shows the index varied between 0 and around 25 during 2006.

So, what the index really tells us, is that the commodities prices tracked by the index grew 60% between April 2009 and April 2010, but only 26% between May 2009 and May 2010. Another way of expressing this, far more fairly, would be:

“Commodity Price Growth reduces to 26% p.a.”

That’s the real story. There’s a huge difference between a growth-rate index and a price index. Reporting the change in a growth-rate index (itself a percentage) by dividing and quoting this as another percentage makes no sense. If inflation goes from 5% one month to 4%, is inflation -20%? No.

Rein in your sensationalist reporters, Bloomberg.


Literature Review on Oil Depletion

May 3, 2010

I’ve spent some of the last few months studying oil depletion. It’s less about models and more about strategy, information and international politics. The Geologists are mainly pessimistic, predicting peak oil soon. The economists are more optimistic, maybe because they think you can throw money at any economic problem (i.e. by higher prices) and get more oil. Ultimately though, once non-OPEC oil is gone (which must happen first, because OPEC hold much more reserves), OPEC hold all the cards and nobody can predict what they’ll do. Read my full PDF.


Commodity Futures Curves Visualisation – “The Movie”

March 18, 2010

My current project needs a good visualisation of various commodity futures curves and their evolution over time. I’ve seen dozens of ‘Snapshots in time’ pictures in various sources, but I knew there was a better way. Some coding in Matlab has enabled me to observe the futures curves evolving in time, rendered as a movie.

Below are some samples, for the period 2004-2009. The lefthand figure is the futures curve, the righthand figure is the price history of the front-most contract (usually as close as we get to ‘spot’ price). Linear interpolation has been used if some months are not traded.

Please comment if you can’t view the movies (Apple people, can you see them?) or want other commodities.

NOTE : Download the movies for much better quality.

Energy Commodities

WTI Crude Oil, ‘CL’ 2004-2010 (download, 28Mb) / 1985-2005 (download, 100Mb)

Nymex Natural Gas, ‘NG’ (download movie, 27Mb)

Metals

Copper, High Grade, ‘HG’. (download movie, 28Mb)

Agriculturals

Wheat, #2 Soft Soft Red Winter, ‘W’. (download movie, 28Mb)

Cocoa, LIFFE (download movie, 29Mb)


Recommended Paper – Commodities Futures Curves

March 15, 2010

A paper (PDF) written back in 1991 by Jacques Gabillon is an excellent introduction to futures curves. He describes the general features of futures curves and specifically those in the oil market, such as the typical ways they change over time, the concepts of ‘backwardation’ and ‘contango’, and the term structure of volatility. He compares the futures curve with the term structure of interest rates.

He then goes on to examine simple models for oil prices using a single SDE (stochastic differential equation) to describe the short-term movements, and clearly explains the role of convenience yield and why cannot be a constant in time and across all maturities. He gradually adds necessary features to a simple oil model to captures more and more empirically observed features of the futures curve. This culminates in his proposed 2-factor model, now known as the Gabillon Model, having both the short term and (unobserved) long term price of oil represented as an (arithmetic) Brownian Motion.

As well as an excellent introduction to his own model, Gabillon’s paper (freely available here (PDF), no login required) is a great introduction to futures curves. It is as relevant today as when it was written, and the principles apply to all commodities, not just oil.

Reference:

Gabillon, J. 1991. The term structures of oil futures prices. Working Paper. Oxford Institute for Energy Studies. www.oxfordenergy.org/pdfs/WPM17.pdf


Commodity Futures Curve Interpolation – Parallel Matlab

March 7, 2010

Matlab 2010a has been recently released, and finally the student version has the Parallel Computing Toolbox available for purchase as an option. I’ve been waiting to play with this for several years.

I document in the PDF below my initial experiments with the toolbox. Specifically, I look into a simple, parallelizable task, that of interpolating commodity futures curves, when we often want to interpolate many days’ curves. Each can be processed independently of the others.

I document a basic way of parallelising this algorithm in Matlab, what kind of speedups to expect using the ‘parfor’ syntax, and the risk of significant slowdown if care is not taken.

I also manage to make the software completely backwards compatible, so it can be shared with those who don’t have the parallel computing toolbox.

Finally, I show a way of making intelligent libraries. For small sized problems, they will execute in serial, because the parallel method imposes big overheads. For larger problem sizes, they will switch to parallel mode.

The PDF includes all the nececessary Matlab software (‘codes’) for others to duplicates these results.

Any comments would be very welcome. I am going to be using the parallel features of Matlab more in coming weeks. It’s good practise, as we seem mainstream 4, 8 and soon 12 core desktops. I have also been experimenting with using CUDA on Matlab, and can recommend the ‘gp-you’ toolbox, which makes CUDA on Matlab easy to learn. It’s yet to be a full-power toolbox though, and is missing things like random number generation on the GPU, essential for monte-carlo simulations.

Download full article (PDF)


Key Papers in Commodities Finance

February 25, 2010

I have compiled a list of some of the most important and influential papers and models, as an introduction to commodities finance or for those involved in modelling commodities (oil, gas, electricity, metals and agricultural products). It is not exhaustive; it is intended as a guide to ‘where to get started’.

Version 1.4 (PDF)