Beware the Confident Counterfactual

Did you anticipate the Syrian uprising that began in 2011? What about the Tunisian, Egyptian, and Libyan uprisings that preceded and arguably shaped it? Did you anticipate that Assad would survive the first three years of civil war there, or that Iraq’s civil war would wax again as intensely as it has in the past few days?

All of these events or outcomes were difficult forecasting problems before they occurred, and many observers have been frank about their own surprise at many of them. At the same time, many of those same observers speak with confidence about the causes of those events. The invasion of Iraq in 2003 surely is or is not the cause of the now-raging civil war in that country. The absence of direct US or NATO military intervention in Syria is or is not to blame for continuation of that country’s civil war and the mass atrocities it has brought—and, by extension, the resurgence of civil war in Iraq.

But here’s the thing: strong causal claims require some confidence about how history would have unfolded in the absence of the cause of interest, and those counterfactual histories are no easier to get right than observed history was to anticipate.

Like all of the most interesting questions, what causality means and how we might demonstrate it will forever be matters for debate—see here on Daniel Little’s blog for an overview of that debate’s recent state—but most conceptions revolve around some idea of necessity. When we say X caused Y, we usually mean that had X not occurred, Y wouldn’t have happened, either. Subtler or less stringent versions might center on salience instead of necessity and insert a “probably” into the final phrase of the previous sentence, but the core idea is the same.

In nonexperimental social science, this logic implicitly obliges us to consider the various ways history might have unfolded in response to X’ rather than X. In a sense, then, both prediction and explanation are forecasting problems. They require us to imagine states of the world we have not seen and to connect them in plausible ways to to ones we have. If anything, the counterfactual predictions required for explanation are more frustrating epistemological problems than the true forecasts, because we will never get to see the outcome(s) against which we could assess the accuracy of our guesses.

As Robert Jervis pointed out in his contribution to a 1996 edited volume on counterfactual thought experiments in world politics, counterfactuals are (or should be) especially hard to construct—and thus causal claims especially hard to make—when the causal processes of interest involve systems. For Jervis,

A system exists when elements or units are interconnected so that the system has emergent properties—i.e., its characteristics and behavior canot be inferred from the characteristics and behavior of the units taken individually—and when changes in one unit or the relationship between any two of them produce ramifying alterations in other units or relationships.

As Jervis notes,

A great deal of thinking about causation…is based on comparing two situations that are the same in all ways except one. Any differences in the outcome, whether actual or expected…can be attributed to the difference in the state of the one element…

Under many circumstances, this method is powerful and appropriate. But it runs into serious problems when we are dealing with systems because other things simply cannot be held constant: as Garret Hardin nicely puts it, in a system, ‘we can never do merely one thing.’

Jervis sketches a few thought experiments to drive this point home. He has a nice one about the effects of external interventions on civil wars that is topical here, but I think his New York traffic example is more resonant:

In everyday thought experiments we ask what would have happened if one element in our world had been different. Living in New York, I often hear people speculate that traffic would be unbearable (as opposed to merely terrible) had Robert Moses not built his highways, bridges, and tunnels. But to try to estimate what things would have been like, we cannot merely subtract these structures from today’s Manhattan landscape. The traffic patterns, the location of businesses and residences, and the number of private automobiles that are now on the streets are in significant measure the product of Moses’s road network. Had it not been built, or had it been built differently, many other things would have been different. Traffic might now be worse, but it is also possible that it would have been better because a more efficient public transportation system would have been developed or because the city would not have grown so large and prosperous without the highways.

Substitute “invade Iraq” or “fail to invade Syria” for Moses’s bridges and tunnels, and I hope you see what I mean.

In the end, it’s much harder to get beyond banal observations about influences to strong claims about causality than our story-telling minds and the popular media that cater to them would like. Of course the invasion of Iraq in 2003 or the absence of Western military intervention in Syria have shaped the histories that followed. But what would have happened in their absence—and, by implication, what would happen now if, for example, the US now re-inserted its armed forces into Iraq or attempted to topple Assad? Those questions are far tougher to answer, and we should beware of anyone who speaks with great confidence about their answers. If you’re a social scientist who isn’t comfortable making and confident in the accuracy of your predictions, you shouldn’t be comfortable making and confident in the validity of your causal claims, either.

Leave a comment

11 Comments

  1. I think Libya was pretty predictable, but I could never have predicted the Syrian uprising, and I thought the Tunisian and Egyptian uprisings were remarkably difficult to predict. Iraq’s new civil war was pretty easy to predict, but I didn’t see Assad surviving into this year until 2013.

    One especially difficult problem is that if the actual cause of a result didn’t occur, it may have been that another cause would have caused the same result.

    Reply
  2. Also, I wonder which people and models correctly predicted that the Arab Spring would not spread to Morocco, Jordan, and Algeria.

    Reply
    • Grant

       /  June 15, 2014

      Of them all, Algeria might make the most sense because of its recent history. People might not like the situation too much, but who wants a return to the 1990s?

      Anyway, it seems to be a trend that ethnically-dominated governments at least help along civil wars. If all the people in Syria were Sunni Arabs, or all the people in Iraq were Shia Arabs, would it have been civil war or revolution and counterrevolution?

      Reply
    • larrydunbar

       /  June 16, 2014

      But I think the point of the article is that who’s to say it didn’t spread to Morocco, Jordan, and Algeria?

      It may be true that the Arab Spring did spread to all those places, simply other bridges and tunnels were built that led to different outcomes, at least to what the casual observer might notice.

      I know, from talking to a person in Jordan, that the Arab spring was there and had manifested itself into an issue of water rights, before it fell off of my radar screen.

      Has anyone heard what’s going on with the arm’s race between Morocco and Algeria? No? But I bet if an armed conflict breaks out between the two it will be something like: Al Qaeda is using Morocco as a base to launch attacks into Algeria, or some other nonsense.

      Reply
  3. Howie ulfelder

     /  June 15, 2014

    Absolutely imperative reading for all my confident friends but least likely to be read by those who need it the most.

    Reply
  1. 22nd of June: Rebrand | Too Much Time
  2. Scientific Updating as a Social Process | Dart-Throwing Chimp
  3. The Ethics of Political Science in Practice | Dart-Throwing Chimp
  4. We Are All Victorians | Dart-Throwing Chimp
  5. IR Theories of Future Past, Part 2 | sicherheitspolitik-blog.de
  6. The Ghosts of Wu Chunming’s Past, Present, and Future | Dart-Throwing Chimp

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: