So many choices (but none of them are good) February 14, 2012 at 7:01 am

No, dear reader, not my Valentine’s day, but rather macroeconomic models. Volker Wieland and Maik Wolters has a nice post on VoxEU where they look at the performance of a goodly number of the leading macroeconomic models. This picture in particular struck me:

Macroeconomic model performance

In other words, of the models studied, none – zero, nada, niente – predicted anything like the recession we got, and all the models predicted a materially stronger and swifter recovery than we got. Honestly given this performance shouldn’t there be rather more wailing and knashing of teeth from the economics profession than (with a few honourable exceptions) we have seen?

6 Responses to “So many choices (but none of them are good)”

  1. […] So many choices (but none of them are good) Deus Ex Macchiato […]

  2. And yet people take the most opinionated stances about macro. I don’t think that’s a coincidence.

  3. Follow link for my forecast based on “animal spirits.”
    cheers,
    benign

  4. FYI, the NY Fed published a study recently that described how badly the Fed’s models are working.

  5. Dr. Murphy, I can’t comment on your old posts, although there are ~5 that I’d like to say something about. Here’s one example:

    http://blog.rivast.com/?p=73

    I think you are confusing probability and statistics. Kolmogorov axioms have nothing to do with repeated experiments. Rather, they formulate the (previously specious/murky/vague) idea of probability in precise terms. These days mathematicians refer to probability theory as “merely” the theory of finite, measurable spaces.

    The problem of measure is to assign sizes to sets, sensibly. You may be familiar with some of the problems with sets. one is that the reals are uncountable (so is the interval [0,1]) — so how can you really say how “large” bits of them are? In the probabilistic case, additional coherence is required because the size of the “universe” under consideration is constant.

    A more practical example is the Drake Equation from astronomy. Someone (presumably someone whose surname began with D) decided that it would be “reasonable” to assume that all conditional probabilities and all marginals are known uniform, because the assumption looks good on paper. From this measure the probability of intelligent extraterrestrial life is derived almost trivially.

    (More complicated but still in astronomy: a paper was recently mentioned by the science tweeter @sc_k pointing out similar incoherence in calculations by inflationary cosmologists.)

    These completely unjustified assumptions should ring bells in finance. More often I believe it has to do with sloppiness committed in the name of “We want to put on a trade NOW, can’t you just tell me what the risk is?” as if “risk” could be characterised by a few numbers, and so forth. Volatility measurements based on the past few months’ 10-minute price changes are perfectly reasonable for some things, but a move of 25-sigma by this measure does not constitute a “once-in-a-10^137” event.

    Anyway, those statistics have little to do with Kolmogorov, and there is actually another chap (whose name escapes me at the moment — you can find a discussion of him on Andrew Gelman’s blog) who really made probability theory respectable.

    ——-

    I recently found out that Mandelbrot is considered “a notorious non-mathematician” by, um, “higher” (I guess) mathematicians. A Taleb if you will.

  6. […] – Felix Salmon (More foggy forecasts…) Why we can’t believe the Fed – CFR (And even more foggy forecasts…) So many choices – Deus Ex […]