From Thailand, Evidence of Coups’ Economic Costs

Last year, I used this space to report a statistical analysis I’d done on the impact of successful coups on countries’ economic growth rates. Bottom line: they usually hurt. As summarized in this chart from a post at FiveThirtyEight, my analysis indicated that, on average, coups dent a country’s economy by a couple of percentage points in the year they happen and another point the year after. Those are not trivial effects.

What makes this question so tricky to analyze is the possibility of a two-way relationship between these things. It’s not hard to imagine that coups might damage national economies, but it’s also likely that countries suffering slower growth are generally more susceptible to coups. With country-year data and no experimental controls, we can’t just run a model with growth as the dependent variable and the occurrence of a coup as a predictor and expect to get a reliable estimate of the former on the latter.

In my statistical analysis, I tried to deal with this problem by using coarsened exact matching to focus the comparison on sets of country-years with comparable coup risk in which coups did or did not happen. I believe the results are more informative than what we’d get from a pooled sample of all country-years, but they certainly aren’t the last word. After all, matching does not magically resolve deeper identification problems, even if it can help.

Under these circumstances, a little process-tracing can go a long way. If we look at real-world cases and see processes linking the “treatment” (coups) to the hypothesized effect (economic damage), we bolster our confidence that the effect we saw in our statistical analysis is not ephemeral.

Here, the recent coup in Thailand is serving up some intriguing evidence. In the past week, I have seen two reports  identifying specific ways in which the coup itself, and not the instability that preceded and arguably precipitated it, is damaging Thailand’s economy. First, I saw this Reuters story (emphasis mine):

Thai officials said on Tuesday that the mass departure of Cambodian laborers would dent the economy as thousands more migrant workers, fearing reprisals from the new military government, poured across the border.

Around 170,000 Cambodian workers have headed home in the past week, according to the International Organization for Migration (IOM), although the exodus is now slowing. Many left after hearing rumors that Thailand’s junta was bent on cracking down on illegal migrants.

Then I saw this tidbit in a Credit Suisse analysis shared on Twitter by Andrew Marshall (emphasis mine):

The coup and martial laws have produced stronger negative impact on Thai tourism, worsening the 2014 earnings outlook and could affect the magnitude of recovery anticipated for 2015…

Based on our conversations with [Airports of Thailand Plc], the coup…appears to have had a stronger impact on its international passenger volumes than the political conflicts… While the coup has restored peace in Bangkok and Thailand, and comforted the Thai people [!], we reckon tourists may take this more negatively and have chosen to go to other destinations.

In a country where international tourism contributes nearly 10 percent of the gross domestic product (here), that impact is a serious issue.

What’s important about both of these reports for the question at hand is the explicit connection they make between the occurrence of the coup and the economy-damaging behavior that followed. To me, these look like consequences rather than coincidences. Neither report definitively proves that the occurrence of a coups usually has an independent, negative effect on a country’s economic growth rate, of course. But they do make me more confident that the effect I saw in my statistical analysis is not just an artifact of some deeper forces I failed to consider.

China and Russia and What Could Have Happened

Twenty five years ago, I was strolling down Leningrad’s main drag, Nevsky Prospekt, with a clutch of other American undergraduates who had recently arrived for two months of intensive language study when Professor Edna Andrews dashed up to us with the news. “They’re shooting them,” she said (or something like it—who can trust a 25-year-old memory of a speech fragment?) with obvious agitation. “They’re shooting the students in Tiananmen Square!”

Had Edna not given us that news, we probably wouldn’t have heard it, or at least not until we got home. In 1989, glasnost’ had already come to the USSR, but that didn’t mean speech was free. State newspapers were still the only ones around, at least for those of us without connections to the world of samizdat. Some of those newspapers were more informative than others, but the limits of political conversation were still clearly proscribed. The Internet didn’t exist, and international calls could only be made by appointment from state-run locations with plastic phones in cubicle-like spaces and who-knows who listening while you talked. Trustworthy information still only trickled through a public sphere mostly bifurcated between propaganda and silence.

What’s striking to me in retrospect is how differently things could have turned out in both countries. When she gave us the news about Tiananmen, Edna was surely agitated because it involved students like the ones she taught being slaughtered. I suspect she was also distressed, though, because at the time it was still easy to imagine something similar happening in the USSR, perhaps even to people she knew personally.

In 1989, politics had already started to move in the Soviet Union, but neither democratization nor disintegration was a foregone conclusion. That spring, citizens had picked delegates to the inaugural session of the Congress of People’s Deputies in elections that were, at the time, the freest the USSR had ever held. The new Congress’ sessions were shown on live television, and their content was stunning. “Deputies from around the country railed against every scandal and shortcoming of the Soviet system that could be identified,” Thomas Skallerup and James P. Nichol describe in their chapter for the Library of Congress’ Russia country study. “Speakers spared neither Gorbachev, the KGB, nor the military.”

But the outspokenness of those reformist deputies belied their formal power. More than 80 percent of the Congress’ deputies were Communist Party members, and the new legislative body the deputies elected that summer, the Supreme Soviet of the USSR, was stuffed with “old-style party apparatchiks.” Two years later, reactionaries inside the government mounted a coup attempt in which President Gorbachev was arrested and detained for a few days and tanks were deployed on the streets of Moscow.

Tank near Red Square on 19 August 1991. © Anatoly Sapronyenkov/AFP/Getty Images

That August Putsch looks a bit clowny with hindsight, but it didn’t have to fail. Likewise, the brutal suppression of China’s 1989 uprising didn’t have to happen, or to succeed when it did. In a story published this week in the New York Times, Andrew Jacobs and Chris Buckley describe the uncertainty of Chinese policy toward the uprising and the disunity of the armed forces tasked with executing it—and, eventually, the protesters in Tiananmen Square.

“At the time,” Jacobs and Buckley write, “few in the military wanted to take direct responsibility for the decision to fire on civilians. Even as troops pressed into Beijing, they were given vague, confusing instructions about what to do, and some commanders sought reassurances that they would not be required to shoot.” Seven senior commanders signed a petition calling on political leaders to withdraw the troops. Those leaders responded by disconnecting many of the special phones those commanders used to communicate with each other. When troops were finally given orders to retake the square “at any cost,” some commanders ignored them. At least one pretended that his battalion’s radio had malfunctioned.

As Erica Chenoweth and Maria Stephan show in their study of civil resistance, nonviolent uprisings are much more likely to succeed when they prompt defections by security forces. The Tiananmen uprising was crushed, but history could have slipped in many other directions. And it still can.

Conflict Events, Coup Forecasts, and Data Prospecting

Last week, for an upcoming post to the interim blog of the atrocities early-warning project I direct, I got to digging around in ACLED’s conflict event data for the first time. Once I had the data processed, I started wondering if they might help improve forecasts of coup attempts, too. That train of thought led to the preliminary results I’ll describe here, and to a general reminder of the often-frustrating nature of applied statistical forecasting.

ACLED is the Armed Conflict Location & Event Data Project, a U.S. Department of Defense–funded, multi-year endeavor to capture information about instances of political violence in sub-Saharan Africa from 1997 to the present.ACLED’s coders scan an array of print and broadcast sources, identifiy relevant events from them, and then record those events’ date, location, and form (battle, violence against civilians, or riots/protests); the types of actors involved; whether or not territory changed hands; and the number of fatalities that occurred. Researchers can download all of the project’s data in various formats and structures from the Data page, one of the better ones I’ve seen in political science.

I came to ACLED last week because I wanted to see if violence against civilians in Somalia had waxed, waned, or held steady in recent months. Trying to answer that question with their data meant:

  • Downloading two Excel spreadsheets, Version 4 of the data for 1997-2013 and the Realtime Data file covering (so far) the first five months of this year;
  • Processing and merging those two files, which took a little work because my software had trouble reading the original spreadsheets and the labels and formats differed a bit across them; and
  • Subsetting and summarizing the data on violence against civilians in Somalia, which also took some care because there was an extra space at the end of the relevant label in some of the records.

Once I had done these things, it was easy to generalize it to the entire data set, producing tables with monthly counts of fatalities and events by type  for all African countries over the past 13 years. And, once I had those country-month counts of conflict events, it was easy to imagine using them to try to help forecast of coup attempts in the world’s most coup-prone region. Other things being equal, variations across countries and over time in the frequency of conflict events might tell us a little more about the state of politics in those countries, and therefore where and when coup attempts are more likely to happen.

Well, in this case, it turns out they don’t tell us much more. The plot below shows ROC curves and the areas under those curves for the out-of-sample predictions from a five-fold cross-validation exercise involving a few country-month models of coup attempts. The Base Model includes: national political regime type (the categorization scheme from PITF’s global instability model applied to Polity 3d, the spell-file version); time since last change in Polity score (in days, logged); infant mortality rate (relative to the annual global median, logged); and an indicator for any coup attempts in the previous 24 months (yes/no). The three other models add logged sums of counts of ACLED events by type—battles, violence against civilians, or riots/protests—in the same country over the previous three, six, or 12 months, respectively. These are all logistic regression models, and the dependent variable is a binary one indicating whether or not any coup attempts (successful or failed) occurred in that country during that month, according to Powell and Thyne.

ROC Curves and AUC Scores from Five-Fold Cross-Validation of Coup Models Without and With ACLED Event Counts

ROC Curves and AUC Scores from Five-Fold Cross-Validation of Coup Models Without and With ACLED Event Counts

As the chart shows, adding the conflict event counts to the base model seems to buy us a smidgen more discriminatory power, but not enough to have confidence that they would routinely lead to more accurate forecasts. Intriguingly, the crossing of the ROC curves suggests that the base model, which emphasizes structural conditions, is actually a little better at identifying the most coup-prone countries. The addition of conflict event counts to the model leads to some under-prediction of coups in that high-risk set, but the balance tips the other way in countries with less structural vulnerability. In the aggregate, though, there is virtually no difference in discriminatory power between the base model and the ones that at the conflict event counts.

There are, of course, many other ways to group and slice ACLED’s data, but the rarity of coups leads me to believe that narrower cuts or alternative operationalizations aren’t likely to produce stronger predictive signals. In Africa since 1997, there are only 36 country-months with coup attempts, according to Powell and Thyne. When the events are this rare and complex and the examples this few, there’s really not much point in going beyond the most direct measures. Under these circumstances, we’re unlikely to discover finer patterns, and if we do, we probably shouldn’t have much confidence in them. There are also other models and techniques to try, but I’m dubious for the same reasons. (FWIW, I did try Random Forests and got virtually identical accuracy.)

So those are the preliminary results from this specific exercise. (The R scripts I used are on Github, here). I think those results are interesting in their own right, but the process involved in getting to them is also a great example of the often-frustrating nature of applied statistical forecasting. I spent a few hours each day for three days straight getting from the thought of exploring ACLED to the results described here. Nearly all of that time was spent processing data; only the last half-hour or so involved any modeling. As is often the case, a lot of that data-processing time was really just me staring at my monitor trying to think of another way to solve some problem I’d already tried and failed to solve.

In my experience, that kind of null result is where nearly all statistical forecasting ideas end. Even when you’re lucky enough to have the data to pursue them, few of your ideas pan out. But panning is the right metaphor, I think. Most of the work is repetitive and frustrating, but every so often you catch a nice nugget. Those nuggets tempt you to keep looking for more, and once in a great while, they can make you rich.

Ripple Effects from Thailand’s Coup

Thailand just had another coup, its first since 2006 but its twelfth since 1932. Here are a few things statistical analysis tells us about how that coup is likely to reverberate through Thailand’s economy and politics for the next few years.

1. Economic growth will probably suffer a bit more. Thailand’s economy was already struggling in 2014, thanks in part to the political instability to which the military leadership was reacting. Still, a statistical analysis I did a few years ago indicates that the coup itself will probably impose yet more drag on the economy. When we compare annual GDP growth rates from countries that suffered coups to similarly susceptible ones that didn’t, we see an average difference of about 2 percentage points in the year of the coup and another 1 percentage point the year after. (See this FiveThirtyEight post for a nice plot and discussion of those results.) Thailand might find its way to the “good” side of the distribution underlying those averages, but the central tendency suggests an additional knock on the country’s economy.

2. The risk of yet another coup will remain elevated for several years. The “coup trap” is real. Countries that have recently suffered successful or failed coup attempts are more likely to get hit again than ones that haven’t. This increase in risk seems to persist for several years, so Thailand will probably stick toward the top of the global watch list for these events until at least 2019.

3. Thailand’s risk of state-led mass killing has nearly tripled…but remains modest. The risk and occurrence of coups and the character of a country’s national political regime feature prominently in the multimodel ensemble we’re using in our atrocities early-warning project to assess risks of onsets of state-led mass killing. When I recently updated those assessments using data from year-end 2013—coming soon to a blog near you!—Thailand remained toward the bottom of the global distribution: 100th of 162 countries, with a predicted probability of just 0.3%. If I alter the inputs to that ensemble to capture the occurrence of this week’s coup and its effect on Thailand’s regime type, the predicted probability jumps to about 0.8%.

That’s a big change in relative risk, but it’s not enough of a change in absolute risk to push the country into the end of the global distribution where the vast majority of these events occur. In the latest assessments, a risk of 0.8% would have placed Thailand about 50th in the world, still essentially indistinguishable from the many other countries in that long, thin tail. Even with changes in these important risk factors and an ongoing insurgency in its southern provinces, Thailand remains in the vast bloc of countries where state-led mass killing is extremely unlikely, thanks (statistically speaking) to its relative wealth, the strength of its connection to the global economy, and the absence of certain markers of atrocities-prone regimes.

4. Democracy will probably be restored within the next few years… As Henk Goemans and Nikolay Marinov show in a paper published last year in the British Journal of Political Science, since the end of the Cold War, most coups have been followed within a few years by competitive elections. The pattern they observe is even stronger in countries that have at least seven years of democratic experience and have held at least two elections, as Thailand does and has. In a paper forthcoming in Foreign Policy Analysis that uses a different measure of coups, Jonathan Powell and Clayton Thyne see that same broad pattern. After the 2006 coup, it took Thailand a little over a year to get back to a competitive elections for a civilian government under a new constitution. If anything, I would expect this junta to move a little faster, and I would be very surprised if the same junta was still ruling in 2016.

5. …but it could wind up right back here again after that. As implied by nos. 1 and 2 above, however, the resumption of democracy wouldn’t mean that Thailand won’t repeat the cycle again. Both statistical and game-theoretic models indicate that prospects for yet another democratic breakdown will stay relatively high as long as Thai politics remains sharply polarized. My knowledge of Thailand is shallow, but the people I read or follow who know the country much better skew pessimistic on the prospects for this polarization ending soon. From afar, I wonder if it’s ultimately a matter of generational change and suspect that Thailand will finally switch to a stable and less contentious equilibrium when today’s conservative leaders start retiring from their jobs in the military and bureaucracy age out of street politics.

Military Coup in Thailand

This morning here but this afternoon in Thailand, the country’s military leadership sealed the deal on a coup d’etat when it announced via national television that it was taking control of government.

The declaration of martial law that came two days earlier didn’t quite qualify as a coup because it didn’t involve a seizure of power. Most academic definitions of coups have involve (1) the use or threat of force (2) by political insiders, that is, people inside government or state security forces (3) to seize national political power. Some definitions also specify that the putschists’ actions must be illegal or extra-constitutional. The declaration of martial law certainly involved the use or threat of force by political insiders, but it did not entail a direct grab for power and technically was not even illegal.

Today’s announcement checks those last boxes. Frankly, I’m a bit surprised by this turn of events, but not shocked. In my statistical assessments of coup risk for 2014, Thailand ranked 10th, clearly putting it among the highest-risk countries in the world. In December, though, I judged from a distance that the country’s military leadership probably didn’t want to take ownership of this situation unless things got significantly worse:

The big question now is whether or not the military leadership will respond as desired [by anti-government forces angling for a coup]. They would be very likely to do so if they coveted power for themselves, but I think it’s pretty clear from their actions that many of them don’t. I suspect that’s partly because they saw after 2006 that seizing power didn’t really fix anything and carried all kinds of additional economic and reputational costs. If that’s right, then the military will only seize power again if the situation degenerates enough to make the costs of inaction even worse—say, into sustained fighting between rival factions, like we see in Bangladesh right now.

I guess the growing concerns about an impending civil war and economic recession were finally enough to tip military leaders’ thinking in favor of action. Here’s hoping the final forecast I offered in that December post comes true:

Whatever happens this time around, though, the good news is that within a decade or so, Thai politics will probably stabilize into a new normal in which the military no longer acts directly in politics and parts of what’s now Pheu Thai and its coalition compete against each other and the remnants of today’s conservative forces for power through the ballot box.

Ukraine’s Just Coup

As Ukraine’s newly appointed government confronts a deepening separatist challenge in Crimea, Viktor Yanukovych continues to describe his removal from office as a “coup d’etat” (here). According to a recent poll by a reputable firm, roughly one-quarter of Russians agree. A month earlier, 84 percent of respondents in a similar poll saw the protests against Yanukovich as a coup attempt.

But that’s all spin and propaganda, right? Yanukovych is a friend of Moscow’s, which presumably views his ouster as part of a broader Western plot against it, and state-guided Russian media have been peddling this line from the start of the EuroMaidan protests a few months ago.

Well, pedantically, Yanukovych is correct. Academic definitions of coups d’etat generally include four criteria: 1) they replace the chief executive; 2) they do not follow constitutional procedure; 3) they are led or facilitated by political insiders; and 4) they involve the use or threat of force. Sometimes we attach modifiers to signify which political insiders strike the blow—military, palace, parliamentary, or judicial—and the criterion regarding the use or threat of force is often interpreted broadly to include arrest or even credibly menacing statements. When political outsiders topple a ruler, we call it a successful rebellion, not a coup. When political insiders remove a sitting leader by constitutional means, we call it politics.

Ukraine unambiguously satisfies at least a few of these criteria. The sitting chief executive was removed from office in a vote by parliamentarians, who qualify as political insiders. Those parliamentarians were encouraged by a popular uprising that represents a form of coercion. Even if we assume, as I do, that most participants in that uprising would not have physically harmed Yanukovich had they captured him, their forceful attempts to seize and occupy government buildings and their clashes with state security forces are clearly coercive acts.

And, crucially, the vote to remove Yanukovych doesn’t seem to have followed constitutional procedures. Under Articles 108-112 of Ukraine’s constitution (here), there are four ways a sitting president may leave office between elections: resignation, incapacitation, death, and impeachment. None of the first three happened—early rumors to the contrary, Yanukovych has vehemently denied that he resigned—so that leaves the fourth, impeachment. According to Article 111, impeachment must follow a specific set of procedures: Parliament must vote to impeach and then convene a committee to investigate. That committee must investigate and report back to parliament, which must then vote to bring charges. A final vote to convict may only come after receipt of a judgment from the Constitutional Court that “the acts, of which the President of Ukraine is accused, contain elements of treason or other crime.” Best I can tell, though, those procedures were not followed in this case. Instead, parliament simply voted—380 to 0, in a body with 450 seats—to dismiss Yanukovych and then to hand executive authority on an interim basis to its own speaker (here).

The apparent extra-constitutionality of this process gives us the last of the four criteria listed above. So, technically speaking, Yanukovych’s removal checks all of the boxes for what we would conventionally call a coup. We can quibble about how relevant the threat of force was to this outcome, and thus whether or not the label “parliamentary coup” might fit better than plain old coup, but the basic issue doesn’t seem especially ambiguous.

All of this should sound very familiar to Egyptians. Twice in the past three years, they’ve seen sitting presidents toppled by political insiders while protesters massed nearby. In both instances, the applicability of the “coup” label became a point of intense political debate. People cared, in part, because perceptions affect political outcomes, and what we call an event shapes how people perceive it. We shout over each other until one voice finally drowns out the rest, and what that voice says becomes the history we remember. In a world where “the will of the people” is seen by many as the only legitimate source of state authority, a whiff of illegitimacy hangs about “coup” that doesn’t adhere to “revolution.” In a peculiar twist of logic and semantics, many Egyptians insisted that President Morsi’s removal in July 2013 could not have been a coup because millions of people supported it. The end was right, so the means must have been, too. Coup doesn’t sound right, so it couldn’t have been one of those.

It’s easy to deride that thinking from a distance. It’s even easier with the benefit of a hindsight that can take in all the terrible things Egypt’s ruling junta has done since it seized power last July.

Before we sneer too hard at those gullible Egyptian liberals, though, we might pause to consider how we’re now describing events in Ukraine, and why. Most of the people I know personally or follow on social media believe that Yanukovych was a rotten menace whose removal from office was justified by his corruption and, more recently, his responsibility for the use of disproportionate force against activists massed on the Maidan. I agree, and I’m sure the documents his accomplices dumped in the Dnipro River on the way out of town will only clarify and strengthen that impression. Yanukovych’s election win in 2010 and his continuing popularity among a large (but dwindling) segment of the population weighed in his favor before 19-20 February, but the shooting to death of scores of unarmed or crudely armed protesters undoubtedly qualifies as the sort of crime that should trigger an impeachment and might even win a conviction. That is, those shootings qualify as an impeachable offense, but impeachment is not what happened.

As moral beings, we can recognize all of those things, and we can and should weigh them in our judgments about the justice of what’s transpired in Ukraine in the past week. Moral and analytical thinking aren’t the same thing, however, and they don’t always point in the same direction, or even occur on the same plane. I’d like to believe that, as analytical thinkers, we’re capable of acknowledging the parallels between Yanukovich’s removal from power and the things we usually call coups without presuming that this acknowledgement negates our moral judgment about the righteousness of that turn of events. Those two streams of thought can and should and inevitably will inform each other, but they don’t have to move deterministically together. Let there be such a thing as a just coup, and let this be an instance of it.

PS. For an excellent discussion of the philosophical issues I gloss over in that final declaration, see Zack Beauchamp’s “The Political Theory Behind Egypt’s Coup” (here).

Cold War Meddling and Coup Attempts

Last April, the American Economic Review published a fascinating paper by some prominent economists that’s only just come onto my radar screen, thanks to a recent Facebook post from Cullen Hendrix. Here’s the abstract:

We provide evidence that increased political influence, arising from CIA interventions during the Cold War, was used to create a larger foreign market for American products. Following CIA interventions, imports from the US increased dramatically, while total exports to the US were unaffected. The surge in imports was concentrated in industries in which the US had a comparative disadvantage, not a comparative advantage. Our analysis is able to rule out decreased trade costs, changing political ideology, and an increase in US loans and grants as alternative explanations. We provide evidence that the increased imports arose through direct purchases of American products by foreign governments.

That’s quite a result—not because we didn’t think stuff like this happened, but because that “stuff” is rarely discussed in academic work on international political economy and even more rarely makes it into our statistical models. We know that powerful states do lots of things to try to shape the world around them, but those things are often hard to observe and even harder to measure, so we usually just leave those “treatments” out of our statistical abstractions of international and domestic politics and hope for the best. (Hello, omitted variable bias!)

But I digress. Where Berger & co. were primarily interested in the economic effects of those interventions, their paper got me thinking about the political ones. Might the episodes of Cold War meddling the authors identify in their data help explain where and when coup attempts happened, too? Theory says “probably,” but not necessarily in a simple way. As Hein Goemans and Nikolay Marinov have argued (here), during the Cold War

The United States and the former colonial powers in Europe had an ambiguous attitude toward coup plots: sometimes helping, sometimes thwarting, and sometimes doing nothing… Because the world was thought to be a chessboard of West vs. East, attitudes toward both the seizure of power by the military and about whether to pressure for elections varied by which side of the ideological conflict the relevant actors took.

Well, was there a clear pattern? Berger & co. have posted extensive replication files (here), and I’ve already got data and scripts to estimate and compare statistical models of coup risk (here), so let’s have a look, starting with a few words on the measures of foreign meddling.

What Berger & co. have compiled are annual, binary indicators of CIA and KGB influence in domestic politics for all countries of the world except the USA and USSR during the period 1947-1990. By “influence,” they mean “periods in which a leader is installed or supported” by one or the other agency. For example, they tag Chile with a 1 (yes) in the column for U.S. influence from 1964, when the CIA supported Eduardo Frei’s successful election campaign and then went on to back various right-wing groups, until 1970, when the presidency passed to Salvador Allende, of whom the U.S. government was not so fond.

Instead of trying to assess the importance of these influence measures by estimating a model and looking at coefficients and p-values, I took some advice from Mike Ward & co. (here) and asked if these variables could help us predict coup attempts. If these episodes of CIA and KGB meddling had much effect on coup risk, then their addition to a base model of coup risk should noticeably improve that model’s predictive power.

To see if they do, I used a 10-fold cross-validation process to get out-of-sample estimates of coup risk from models without and with Berger & co.’s measures of CIA and KGB influence, then compared the accuracy of those two sets of estimates. The base model is the same one I used in my coup forecasts for 2014, and it covers a lot of ground, from national wealth and colonial legacies to political regime type and recent coup activity. Country-years are the units of observation, and the dependent variable in all cases is a binary indicator of whether or not any coup attempts (successful or failed) occurred during that calendar year. Except for the marker for election years, all covariates—including the measures of U.S. and Soviet influence—were lagged one year.

Here’s a plot of the ROC curves for out-of-sample estimates of coup risk from models without (black) and with (red) those intervention indicators. The numbers reported in the bottom right-hand corner of the plot are the areas under those curves (AUC), and bigger is better. As you can see, the addition of these lagged indicators of periods of CIA and KGB influence has no real effect on the model’s predictive power, suggesting that these variables don’t help explain the location and timing of coups, at least not in the context of this model.

A Comparison of the Predictive Power of Coup Risk Models Without and With Measures of CIA and KGB Influence

A Comparison of the Predictive Power of Logistic Regression Models of Coup Risk Without (black) and With (red) Lagged Indicators of CIA and KGB Influence

After seeing those results, I wondered if the effects of U.S. and Soviet influence on coup risk might depend on some of the other things in my model, like political regime type or the occurrence of elections. To allow for contingent effects without specifying exactly what those contingencies might be, I re-ran the analysis using Random Forests instead of logistic regression. The plot below shows the results. Again, nothing doing.

A Comparison

A Comparison of the Predictive Power of Random Forests of Coup Risk Without (black) and With (red) Lagged Indicators of CIA and KGB Influence

It would be easy to look at those plots and conclude that CIA and KGB meddling didn’t play much of a role in coups during the Cold War after all. It’d be easy, but it’d also be wrong.

One of the great things about the paper that set off this whole exercise is that the authors extensively document their data, to include short descriptions of the forms of U.S. and Soviet influence they observe and the sources from which that information came. When we take a closer look at those descriptions (“Summary_of_Interventions.pdf”), it becomes clearer that the source of my null result isn’t the fact that the CIA and KGB weren’t in the business of backing or thwarting coups. Instead, the issue is that the effects often flowed in the opposite direction. That is, coups and other forms of meddling in the selection of national leaders were often what started these episodes of influence in the first place. Influence doesn’t cause coups; coups cause influence.

Once we realize this, it’s less surprising to discover that coups were actually somewhat less likely during periods of CIA or KGB influence than not. When I estimate a logistic regression model using all of the available data (1960-1990), the coefficients on the lagged indicators of CIA and KGB influence are both negative and not tiny: -0.3 and -1.0, respectively, with standard errors of 0.2 and 0.5. If you think those agencies might have helped protect their clients from domestic rivals, that result makes sense.

Ultimately, there are two lessons here, one substantive and one methodological. On the substantive side, this exercise reaffirms the knowledge that the great powers of the Cold War era had significant sway over politics within many other states, including the selection and survival of their leaders. Of course, we didn’t need a statistical analysis to tell us that; we only needed to review the evidence Berger & co. have compiled on the way to their finding about how those efforts affected international trade.

On the methodological side, the counter-intuitive findings from the analysis described in this blog post are a useful reminder that we shouldn’t stop thinking when we hit Enter on our statistical estimations. When interpreting results like these, we have to think carefully about what is and isn’t being measured. The models we can specify with data and methods at hand don’t always match the ideas in our heads, and it’s on us to keep the two straight.

Coup Forecasts for 2014

This year, I’ll start with the forecasts, then describe the process. First, though, a couple of things to bear in mind as you look at the map and dot plot:

  1. Coup attempts rarely occur, so the predicted probabilities are all on the low side, and most are approximately zero. The fact that a country shows up in dark red on the map or ranks high on an ordered list does not mean that we should anticipate a coup occurring there. It just means that country is at relatively high risk compared to the rest of the world. Statistically speaking, the safest bet for any country almost any year is that a coup attempt won’t occur. The point of this exercise is to try to get a better handle on where the few coup attempts we can expect to see this year are most likely to happen.
  2. These forecasts are estimates based on noisy data, so they are highly imprecise, and small differences are not terribly meaningful. The fact that one country lands a few notches higher or lower than other on an ordered list does not imply a significant difference in risk.

Okay, now the the forecasts. First, the heat-map version, which sorts the world into fifths. From cross-validation in the historical data, we can expect nearly 80 percent of the countries with coup attempts this year to be somewhere in that top fifth. So, if there are four countries with coup attempts in 2014, three of them are probably in dark red on that map, and the other one is probably dark orange.

forecast.heatmap.2014

Now, a dot plot of the Top 40, which is a slightly larger set than the top fifth in the heat map. Here, the gray dots show the forecasts from the two component models (see below), while the red dots are the unweighted average of those two—what I consider the single-best forecast.

forecast.dotplot.2014

A lot of food for thought in there, but I’m going to leave interpretation of these results to future posts and to you.

Now, on the process: As statistical forecasters are wont to do, I have tinkered with the models again this year. As I said in a blogged research note a couple of weeks ago, this year’s tinkering was driven by a combination of data practicalities and the usual sense of, “Hey, let’s see if we can do a little better this time.”Predictably, though, I also ended up doing things a little different than I’d expected in December. Specifically:

  • I trained and validated the models on an amalgamation of two coup data sets—as described in a November post that showed an animated map of coup attempts worldwide since 1946—instead of just using the Powell and Thyne list. So that map and the bar plots with it should give you a clearer sense of what these forecasts are (and aren’t) trying to anticipate.
  • After waiting for Freedom House to update its Freedom in the World data, which it did a few days ago, I decided to go back to using Polity after all because the forecasts based on it were noticeably more accurate in cross-validation. The models include a categorical measure of regime type based on the Polity scale and a “clock” counting years since the last significant change in that score. I hard-coded updates to those measures, which are much coarser (and therefore easier to update) than the Polity scale or its component variables.
  • As with coup events, I used an amalgamation of GDP growth data from the World Bank and IMF instead of picking one. I also went back to summarizing this feature in the models with a binary indicator for slow growth of less than 2 percent (annual, per capita).
  • Finally, I did not include GDELT summaries in the models because they only slightly improved forecast accuracy, and they did not cover a country of great interest to me (South Sudan). The latter is surely a fixable glitch, but it’s not fixed now, and I really wanted to have a forecast for that particular country in this year’s list for reasons that should now be evident from the results. On the accuracy part, I should note that I’ve only done a little bit of checking, and there are still plenty of ways to try to squeeze more forecasting power out of those data, not the least of them being to build more dynamic models that use monthly instead of annual summaries.

The forecasts are an unweighted average of predicted probabilities from a logistic regression model and a Random Forest that use more or less the same inputs. Both models were trained on data covering the period 1960-2010; applied to data from 2011 to 2013 to assess their predictive performance; and then applied to the newest data to generate forecasts for 2014. Variable selection was based mostly on my prior experience working this problem. As noted above, I did a little bit of model checking—using stratified 10-fold cross-validation—to make sure the process worked reasonably well, and to help choose between some different measures for the same concept. In that cross-validation, the unweighted average got good but not great accuracy scores, with an area under the ROC curve in the low 0.80s. Here are the variables used in the models:

  • Geographic Region. Per the U.S. Department of State (and only in the Random Forest).
  • Last Colonizer. Indicators for former French, British, and Spanish colonies.
  • Country Age. Years since independence, logged.
  • Post-Cold War Period. Indicator marking country-years since 1991, when coup activity has generally slowed.
  • Infant Mortality Rate. Relative to the annual global median, logged, and courtesy of the U.S. Census Bureau. The latest version ends in 2012, so I’ve simply pulled those values forward a year here.
  • Political Regime Type. Four-way categorization based on the Polity scale into autocracies, “anocracies,” democracies, and transitional, collapsed, or occupied cases.
  • Political Stability. Count of years since a significant change in the Polity scale, logged.
  • Political Salience of Elite Ethnicity. Yes or no, per a data set on elite characteristics produced by the Center for Systemic Peace (CSP) for the Political Instability Task Force (PITF), with hard-coded updates for 2013 (no changes). This one is not posted on CSP’s data page and was obtained from PITF and shared with their permission.
  • Violent Civil Conflict. Yes or no, per CSP’s Major Episodes of Political Violence data set (here), with hard-coded updates for 2013 (a few changes).
  • Election Year. Yes-or-no indicator for any national elections—executive, legislative, or constituent assembly—courtesy of the NELDA project, with hard-coded updates for 2012 through 2014 (scheduled).
  • Slow Economic Growth. Yes-or-no indicator for less than 2 percent, as described above.
  • Domestic Coup Activity. Yes-or-no indicator for countries with any attempts in the past 5 years, successful or failed.
  • Regional Coup Activity. A count of other countries in the same region with any coup attempts the previous year, logged.
  • Global Coup Activity. Same as the previous tic, but for the whole world.

All of the predictors are lagged one year except for region, last colonizer, country age, post-Cold War period, and the election-year indicator. The fact that a variable appears on this list does not necessarily mean that it has a significant effect on the risk of any coup attempts. As I said earlier, I drew up a roster of variables to include based on a sense of what might matter (a.k.a., theory) and past experience and did not try to do much winnowing.

If you are interested in exploring the results in more detail or just trying to do this better, you can replicate my analysis using code I’ve put on GitHub (here). The posted script includes a Google Drive link with the requisite data. If you tinker and find something useful, I only ask that you return the favor and let me know. [N.B. As its name implies, the generation of a Random Forest is partially stochastic, so the results will vary slightly each time the process is repeated. If you run the posted script on the posted data, you can expect to see some small differences in the final estimates. I think these small differences are actually a nice representation of the forecasts' inherent uncertainty, so I have not attempted to eliminate it by, for example, setting the random number seed within the R script.]

For the 2013 version of this post, see here. For 2012, here.

UPDATE: In response to a comment, I tried to produce another version of the heat map that more clearly differentiates the quantiles and better reflects the fact that the predicted probabilities for cases outside the top two fifths are all pretty close to zero. The result is shown below. Here, the differences in the shades of gray represent differences in the average predicted probabilities across the five tiers. You can decide if it’s clearer or not.

forecast.grayscalemap.2014

A Coup Lexicon

Reasonable people disagree on the definition and proper usage of the word “coup.” Inspired by a tweet this morning from Erik Meyersson, I thought I’d put together a lexicon to try to clear things up.

  • Coup d’etat: A sudden usurpation of state power by illegal or extra-constitutional means involving the use or threat of force (or, apparently, a new restaurant in Minneapolis).
  • Military coup: Same as above, with elements of state security forces as the usurpers.
  • Executive coup (a.k.a. autogolpe)In democracies, an abrupt change in rules or procedures by the incumbent chief executive that effectively concentrates power in his or her hands and short-circuits electoral competition. See: India in 1975, Peru in 1992, and Russia in 1993.
  • Judicial coup: A decision by an apparently partisan judiciary that removes an elected government from power. See: Egypt in 2012Thailand in 2008, and, according to those bumper stickers you still see in my neighborhood every once in a while, Bush v. Gore.
  • Parliamentary coup: The removal from office of a directly elected president by a legislature using questionable procedures for apparently partisan purposes. See: Paraguay in 2012 and, for a dog that growled but didn’t quite bark, the United States in 1999.
  • Merci beaucoup: (Fr.) Thank you very much.
  • Mercy bro coup: The successful extrication of a persistently adolescent and self-absorbed young man from an embarrassing situation, probably involving bad beer consumed ironically.
  • Coup de grace: (Fr.) The final blow or stroke that finishes off a sufferer or weakened foe.
  • Coup de grass: What happened in Uruguay a few weeks ago.

What have I missed? Please use the Comments to suggest additions or corrections.

A Research Note on Updating Coup Forecasts

A new year is about to start, and that means it’s time for me to update my coup forecasts (see here and here for the 2013 and 2012 editions, respectively). The forecasts themselves aren’t quite ready yet—I need to wait until mid-January for updates from Freedom House to arrive—but I am making some changes to my forecasting process that I thought I would go ahead and describe now, because the thinking behind them illustrates some important dilemmas and new opportunities for predictions of many kinds of political events.

When it comes time to build a predictive statistical model of some rare political event, it’s usually not the model specification that gives me headaches. For many events of interest, I think we now have a pretty good understanding of which methods and variables are likely to produce more accurate forecasts.

Instead, it’s the data, or really the lack thereof, that sets me to pulling my hair out. As I discussed in a recent post, things we’d like to include in our models fall into a few general classes in this regard:

  • No data exist (fuggeddaboudit)
  • Data exist for some historical period, but they aren’t updated (“HA-ha!”)
  • Data exist and are updated, but they are patchy and not missing at random (so long, some countries)
  • Data exist and are updated, but not until many months or even years later (Spinning Pinwheel of Death)

In the past, I’ve set aside measures that fall into the first three of those sets but gone ahead and used some from the fourth, if I thought the feature was important enough. To generate forecasts before the original sources updated, I either a) pulled forward the last observed value for each case (if the measure was slow-changing, like a country’s infant mortality rate) or b) hand-coded my own updates (if the measure was liable to change from year to year, like a country’s political regime type).

Now, though, I’ve decided to get out of the “artisanal updating” business, too, for all but the most obvious and uncontroversial things, like which countries recently joined the WTO or held national elections. I’m quitting this business, in part, because it takes a lot of time and the results may be pretty noisy. More important, though, I’m also quitting because it’s not so necessary any more, thanks to  timelier updates from some data providers and the arrival of some valuable new data sets.

This commitment to more efficient updating has led me to adopt the following rules of thumb for my 2014 forecasting work:

  • For structural features that don’t change much from year to year (e.g., population size or infant mortality), include the feature and use the last observed value.
  • For variables that can change from year to year in hard-to-predict ways, only include them if the data source is updated in near-real time or, if it’s updated annually, if those updates are delivered within the first few weeks of the new year.
  • In all cases, only use data that are publicly available, to facilitate replication and to encourage more data sharing.

And here are some of the results of applying those rules of thumb to the list of features I’d like to include in my coup forecasting models for 2014.

  • Use Powell and Thyne’s list of coup events instead of Monty Marshall’s. Powell and Thyne’s list is updated throughout the year as events occur, whereas the publicly available version of Marshall’s list is only updated annually, several months after the start of the year. That wouldn’t matter so much if coups were only the dependent variable, but recent coup activity is also an important predictor, so I need the last year’s updates ASAP.
  • Use Freedom House’s Freedom in the World (FIW) data instead of Polity IV to measure countries’ political regime type. Polity IV offers more granular measures of political regime type than Freedom in the World, but Polity updates aren’t posted until spring or summer of the following year, usually more than a third of the way into my annual forecasting window.
  •  Use IMF data on economic growth instead of the World Bank’s. The Bank now updates its World Development Indicators a couple of times a year, and there’s a great R package that makes it easy to download the bits you need. That’s wonderful for slow-changing structural features, but it still doesn’t get me data on economic performance as fast as I’d like it. I work around that problem by using the IMF’s World Economic Outlook Database, which include projections for years for which observed data aren’t yet available and forecasts for several years into the future.
  • Last but not least, use GDELT instead of UCDP/PRIO or Major Episodes of Political Violence (MEPV) to measure civil conflict. Knowing which countries have had civil unrest or violence in the recent past can help us predict coup attempts, but the major publicly available measures of these things are only updated well into the year. GDELT now represents a nice alternative. It covers the whole world, measures lots of different forms of political cooperation and conflict, and is updated daily, so country-year updates are available on January 2. GDELT’s period of observation starts in 1979, so it’s still a stretch to use it models of super-rare events like mass-killing onsets, where the number of available examples since 1979 on which to train is still relatively small. For less-rare events like coup attempts, though, starting the analysis around 1980 is no problem. (Just don’t forget to normalize them!) With some help from John Beieler, I’m already experimenting with adding annual GDELT summaries to my coup forecasting process, and I’m finding that they do improve the model’s out-of-sample predictive power.

In all of the forecasting work I do, my long-term goals are 1) to make the forecasts more dynamic by updating them more frequently (e.g., monthly, weekly, or even daily instead of yearly) and 2) to automate that updating process as much as possible. The changes I’m making to my coup forecasting process for 2014 don’t directly accomplish either of these things, but they do take me a few steps in both directions. For example, once GDELT is in the mix, it’s possible to start thinking about how to switch to monthly or even daily updates that rely on a sliding window of recent GDELT tallies. And once I’ve got a coup data set that updates in near-real time, I can imagine pinging that source each day to update the counts of coup attempts in the past several years. I’m still not where I’d like to be, but I think I’m finally stepping onto a path that can carry me there.

Follow

Get every new post delivered to your Inbox.

Join 7,403 other followers

%d bloggers like this: