Nate Silver on ECRI and economic forecasting

- by New Deal democrat

Nate Silver, the statistics wizard behind the PECOTA ranking system in baseball and aggregator of polls into an almost perfect prediction of the 2012 presidential election, recently wrote The Signal and the Noise: Why so Many Predictions Fail - but Some Don't. Via blogger Ashok yesterday, it turns out that he devoted a chapter to economic forecasting and specifically ECRI's now almost 2 year old "imminent recession" call:

In September 2011, ECRI predicted a near certainty of a “double dip” recession. “There’s nothing that policy makers can do to head it off,” it advised. “If you think this is a bad economy, you haven’t seen anything yet.” In interviews, the managing director of the firm, Lakshman Achuthan, suggested the recession would begin almost immediately if it hadn’t started already. The firm described the reasons for its prediction in this way: “ECRI’s recession call isn’t based on just one or two leading indexes, but on dozens of specialized leading indexes, including the U.S. Long Leading index…. to be followed by downturns in the Weekly Leading Index and other shorter-leading indexes. In fact the most reliable forward looking indicators are now collectively behaving as they did on the cusp of full-blown recessions.” There’s plenty of jargon, but what is lacking in this description is any actual economic substance. Theirs was a story about data – as though data itself caused recessions – and not a story about the economy.

I would hate to be the example Nate Silver decided to use of a statistical faceplant. (Ask Gallup.) If the Hitler "Downfall" parody didn't do so, this is pretty much the official declaration that ECRI blew the call.

Beyond that, I also agree with Ashok's criticism of Silver's analysis above:

I think there’s a disservice in not considering the epistemological impossibility of forecasting recession, quite to the contrary of [the above] passage:
....
... Silver agrees that the best forecaster is one who gets it right. Rationally, the goal of any good forecaster is to be trusted and serve as an important source of information for clients. As far as economic predictions go, the client is of course the free market. Here’s the problem, let’s say I’m a trusted forecaster and I publish a report stating that the American economy will shrink by 4% next quarter. If people trust me, my report will have caused a recession. Why? Because business operations across the country will [scale back now in anticipation of the forecast recession].

Neither I nor any human being in the last 10,000 years had to know what the laws of physics were in order to predict that the sun will rise in the east tomorrow morning. A sufficiently reliable pattern in the data is enough.

The fact is, economic forecasting only works to the extent that the forecaster is the proverbial fly on the wall. Any forecast of any result that depends upon subsequent human behavior is subject to the fact that humans are able to react to the forecast itself. Think of Ebenezer Scrooge and the Ghost of Christmas Future. (Or "Babylon 5:" "Whatever you do, John, DO NOT GO TO ZA'HA'DUM!") If an economic forecaster became widely known as 100% accurate, their forecast would cause people to change their behavior immediately if possible. Thus the forecast of what was to come in 3 or 6 months would likely become a "forecast" of something that happened immediately instead.

So my blogging is ironically subject to the fact that I know that you, my dear reader, are a member of a, shall we say, very select group!

As for ECRI, I think their blown September 2011 call was due to their own human failure to see that the steep sudden fall in the data in summer 2011 was largely a reaction to the debt ceiling debacle. This was a political cause that could be, and was, reversed politically.