Which Past Will the Future Resemble?

Forecasts derived from statistical models depend on the assumption that the future will resemble the past. For modelers, the question is: which past? The time frame(s) we choose to use when developing our models and the ways we deal with history and time in our analyses can have substantial effects on the forecasts we produce, and this is one area where theory is at least as important as coding and statistical skills.

I was reminded of this point yesterday when Chrystia Freeland tweeted a link to the New York Fed’s blog, Liberty Street Economics. There, Ging Cee Ng and Andrea Tambalotti had a post showing (literally, as in with pictures) how the initial choice of a reference period would have affected whether or not standard macroeconomic models could have seen the Great Recession coming. Their concluding paragraph nicely sums up their findings:

Our calculations suggest that the Great Recession was indeed entirely off the radar of a standard macroeconomic model estimated with data drawn exclusively from the Great Moderation [a period of exceptional economic stability running from 1984 to 2007]. By contrast, the extreme events of 2008-09 are seen as far from impossible—if unlikely—by the same model when the shocks hitting the economy are gauged using data from a longer period (third-quarter 1954 to fourth-quarter 2007). These results provide a simple quantitative illustration of the extent to which the Great Moderation, and more specifically the assumption that the tranquil environment characterizing it was permanent, might have led economists to greatly underestimate the possibility of a Great Recession.

For social scientists trying to forecast rare events in the international system–things like civil wars, coups, and mass killings–thinking about historical eras that might be exerting some gravitational pull on patterns in the data often focuses on contrasts between the Cold War and post-Cold War periods. There are a host of ways in which the causes of internal and international crises might have changed when the USSR disintegrated, not the least of them being the end of the proxy wars the rival powers often waged and the coups they sometimes endorsed or promoted.

When developing a model for forecasting, we’re tempted to restrict the analysis to the post-Cold War period with the expectation that the near future will more closely resemble the near past. But what if the two decades following the collapse of the Soviet Union turn out to be the global political equivalent of the Great Moderation? Ng and Tambolotti’s analysis suggests that models estimated from post-Cold War data only would work better as long as the moderation holds (held?), but they would start missing badly when the system shifted back to something more like its long-term state.

Efforts to forecast onsets of mass killing provide a useful example. In the plot of mass-killing onsets by year shown below, the frequency with which these events occur seems to have changed markedly in the post-Cold War period. Over the three decades between decolonization in Africa and the disintegration of the USSR, most years saw two mass-killing onsets, and only a few saw none. Since the spasm of onsets that accompanied the collapse of Communist rule in the early 1990s, however, zero has been the most common occurrence, and no years saw more than a single onset.

Mass-Killing Onsets Worldwide by Year, 1945-2011

Until 2011, that is. For the first time in almost two decades, a single year produced two mass-killing onsets, in Sudan and Syria. Two onsets are not nearly enough to claim that things have changed, but it is enough to get me thinking–especially when it’s also possible that episodes of mass killing may have begun last year in Libya, Yemen, and Egypt.

Maybe 2011 was an anomaly, an unlikely but not impossible year in an ongoing era of less frequent or less intense attacks by states on their civilian populations. Maybe, though, it marked the end of an unusually pacific run, and the world is now sliding back toward its old normal. Until the future actually happens, the best answers we can offer to that question are informed speculation.

In the meantime, modelers looking to assess risks of mass killing in the next several years have to make some practical choices. And, as Ng and Tambalotti’s analysis shows, those choices will have a real effect on the forecasts we produce.