On the Limits of Our Causal Imagination

This morning, while I was driving my boys to school, my 13-year-old son said:

When I was a kid, I thought you controlled the car with the steering wheel. I would see you go like this <pushes arms out> and like this <pulls arms in> and thought that was how you made it go.

What a perfect illustration of how our minds imperfectly construct causality. Sitting in the back seat when he was younger, my son couldn’t see my feet as I drove; he could only see my hands. When he wondered what caused the car to speed up and slow down, he built a complete mental model from observed materials. It didn’t occur to him that I might be doing things he didn’t see—that the real causes of the car’s acceleration and deceleration might lie hidden from his view. Only in retrospect did that idea seem silly. At the time, that mental model made complete sense to him, and he implicitly entrusted his life to it every time he climbed in that car.

Think about that next time you’re trying to explain something as complex as the flow and ebb of a social movement or the collapse of a state.

On the Consequences of Transition Politics for Democratization

In academic work on political development, the term regime transition refers to the period of time between the end of one political regime and the establishment of another. As Guillermo O’Donnell and Philippe Schmitter say on page 6 of their Little Green Book, “It is characteristic of the transition that during it the rules of the political game are not defined. Not only are they in constant flux, but they are usually arduously contested.” Think Tunisia from Ben Ali’s ouster in January 2011 until the convocation of its elected Constituent Assembly in October of that year, or Egypt since Mubarak’s resignation (now 18 months and counting!).

So, we might wonder, does the way that transition unfolds affect the quality and duration of the democracy that ensues? Does it make a difference if, say, this period is characterized by negotiation and compromise instead of tumult and violence? If it’s carefully managed by the remnants of the old regime or driven by outsiders? If democracy is imposed by foreign forces instead of built from within?

According to an interesting paper upon which I recently stumbled—and, I gather, a forthcoming book based on the same research—authors Gary Stradiotto and Sujian Guo conclude the answer to that question is a resounding “yes”:

 The literature offers competing claims among scholars concerning the role the mode of transition plays in influencing post-transitional democracy. The authors reconcile these claims. First, they classify democratic transitions into four transitional modes, and hypothesize that cooperative transitions result in higher levels of democracy that last longer than other transition types. A method to quantitatively test the mode of transition (the independent variable) against democratic quality and longevity (the dependent variables) is developed. The results provide strong confirmation that states that transition through cooperative pacts are associated with higher levels of democracy and a lower risk of reversion compared to other transition types.

The question this paper is trying to answer is a really important one for participants in these transitions, who have to think strategically about how to try to push developments in a particular direction, and for policymakers and activists in other countries, who might wish to influence those trajectories as well.

As careful as the authors are in their analysis, though, and as plausible as their story is, I don’t think their research design succeeds in solving the vexing problems that make it so hard to answer this question with confidence. I’m thinking of two problems in particular.

The first is the problem of confounding factors. The conjecture that modes of transition might have lasting effects on the democracies they produce is rooted in the idea of path dependency, which is just a fancy way of referring to the persistent influence of events or conditions deeper in the past than the moment or period we’re studying. Using this language, the hypothesis Stradiotto and Guo are exploring could be restated as the idea that the survival and quality of democracy after a transition depends, in part, on the form of the politics that occur during the transition process itself.

That statement seems obviously true, and yet it’s devilishly hard to prove. The problem is that transitions don’t occur on blank slates, and the history that preceded the breakdown of the old regime might—really, must—also have some effect on both a) what form the transition takes and b) what happens afterwards. For example, numerous scholars of comparative democratization have argued that the structural features of an authoritarian regime affect the likelihood that the regime will break down, and if it breaks down, that democracy will follow (see here, here, and here). Others emphasize the effects of even deeper forces—things like Jared Diamond’s argument about the persistent influences of climate and geography on political and economic development, or Daron Acemoglu, Simon Johnson, and James Robinson’s claim that institutions imposed at the time of initial colonization have powerfully shaped developmental trajectories right up to the present.

When confounding factors are present, it’s really hard to be sure that the patterns we see are causal and not just coincidental. An analogy might help here. The trajectory of a golf ball, for example, is highly path dependent. Changes in wind speed and direction after the ball is struck will have some effect on where it lands, but, under most conditions, the ball’s flight path is largely predetermined by the way it was struck. If we wanted to explain why balls land where they do, we could analyze the relationship between distinct flight paths–hooks, slices, worm-burners, and such–and landing spots, and we would find a strong association between the two. But, of course, it’s not really the flight path that caused that outcome. Instead, it’s the club selection and swing mechanics that caused that flight path to occur, and it’s the training and experience that caused those swing mechanics and club selection, and so on. If our analysis began the moment the ball left the ground, we would find strong patterns in our data, but we would misunderstand the causes of those patterns.

To look for independent effects of transition modes in the face of this problem, we can’t just pile measures of likely confounding factors into a statistical model and expect to have “controlled” for them. Instead, we have to think more like experimenters. One way to do this is to focus on variation within sets of cases that have similar values on potential confounding factors. Matching before modeling and conditional regression are two ways to do this. Mixed-effects models with cases clustered by likely confounders might work, too, although this could get quite messy if those confounding factors aren’t nested. I suspect the causal-inference pros could suggest many others, and in any case, my point is that, without some more careful structuring of the comparisons, we really can’t tell if variation in the mode of transition is causing variation in outcomes, or if that variation in modes is just symptomatic of deeper differences that would likely have doomed or blessed the ensuing democracy anyway.

The second big problem is selection bias. Stradiotto and Guo limit their study to cases where democracy happened and exclude ones where a transition led to something else. “Excluding cases that never reach a democratic threshold is not problematic,” they argue, “as we are only concerned with understanding how the mode of transition influences the resultant democracy.”

In my view, this isn’t quite right. To fully understand how modes relate to outcomes, we also have to consider transitions that failed to produce democracy. Freek Vermuelen nicely illustrates why in an old post on the Harvard Business Review Blog Network called “Beware the Dangers of Selection Bias”:

Consider, for example, the popular notion that innovation projects require diverse, cross-functional teams. This notion exists because if we analyze some path-breaking innovation projects, we see they were often staffed by such teams. However, it has also been suggested (see for instance the work of Professor Jerker Denrell from Stanford Business School) that diverse, cross-functional teams also often created the biggest failures of all. However, such failures never resulted in any products… Therefore, if we (only) examine the projects which actually resulted in successful innovations, it seems the diverse cross-functional teams did much better. Yet, on average, the homogeneous teams—although not responsible for the few really big inventions—might have done better; always producing a reliable, good set of results.

What Stradiotto and Guo are analyzing is the outcome, conditional on the successful conclusion of the transition. If we’re interested in how the dynamics of the transition process shapes prospects for democratization, though, I think it’s pretty clear that we’ll also want to consider how those dynamics affect whether or not democracy even arises in the first place. Indeed, in an earlier stab at this problem, Gerardo Munck argues that modes of transition have strong effects on both of those stages:

All too often the literature on modes of transition has failed to distinguish between transitions from established regimes and transitions to new regimes and thus reduced the assessment of modes of transition to their impact on the consolidation of democracy. The mode of transition not only affects the consolidation of new regimes but also helps to determine whether the transition is to democracy or some other regime type.

In sum, confounding factors and selection effects make it very hard for us to identify the marginal effects of transition modes on prospects for democratization, and I don’t think Stradiotto and Guo succeed in overcoming these problems in their recent contribution to the literature on this timely question. Perhaps the authors have addressed these issues in their forthcoming book, which I look forward to reading. In the meantime, it’s frustrating that we don’t have much of an answer to this very important question, especially at a moment when so many countries are experiencing these kinds of transitions. Unfortunately, though, I think that’s still where we are.

Follow

Get every new post delivered to your Inbox.

Join 8,706 other followers

%d bloggers like this: