Coup Forecasts for 2013

Last January, I posted statistical estimates of coup risk for 2012 that drew some wider interest after they correctly identified Mali as a high-risk case. Now that the year’s almost over, I thought it would be a good time to assess more formally how those 2012 forecasts performed and then update them for 2013.

So, first things first: how did the 2012 forecasts fare on the whole? Pretty well, actually.

For purposes of these forecasts, a coup is defined as “as a forceful seizure of executive authority and office by a dissident/opposition faction within the country’s ruling or political elites that results in a substantial change in the executive leadership and the policies of the prior regime.” That language comes from Monty Marshall’s Center for Systemic Peace, whose data set on coup events serves as the basis for one of the two models used to generate the 2012 forecasts. Those forecasts were meant to assess the risk of any coup attempts at some point during the calendar year, whether those attempts succeed or fail. They were not meant to anticipate civil wars, non-violent uprisings, voluntary transfers of executive authority, autogolpes, or interventions by foreign forces, all of which are better thought of (and modeled) as different forms of political crisis.

Okay, so by that definition, I see two countries where coup attempts occurred in 2012: Mali (in March) and Guinea-Bissau (in April). As it happens, both of those countries ranked in the top 10 in January’s forecasts—Guinea-Bissau at no. 2 and Mali at no. 10—so the models seem to be homing in on the right things. We can get a more rigorous take on the forecasts’ accuracy with a couple of statistics commonly used to assess models that try to predict binary outcomes like these (either a coup attempt happens or it doesn’t):

  • AUC Score. The estimated area under the Receiver Operating Characteristic (ROC) curve, used as a measure of the ability of a binary classification model to discriminate between positive and negative cases. Specifically, AUC represents the probability that a randomly selected positive case (here, a country-year with coup activity) will have a higher predicted probability than a randomly selected negative case (e.g., country-year with no coup activity). Ranges from 0.5 to 1, with higher values indicating better discrimination.
  • Brier Score. A general measure of forecast performance, defined as the average squared difference between the predicted and observed values. Ranges from 0 to 1, with lower values indicating more accurate predictions.

Assuming that Mali and Guinea-Bissau were the only countries to see coup activity this year, my 2012 coup forecasts get an AUC score of 0.97 and a Brier score of 0.01. Those are really good numbers. Based on my experience trying to forecast other rare political events around the world, I’m pretty happy with any AUC above the low 0.80s and any Brier score that’s better than an across-the-board base-rate forecast. The 2012 coup forecasts surpass both of those benchmarks.

Of course, with just two events in more than 150 countries, these statistics could be very sensitive to changes in the list of coup attempts. Two possible modifications come from Sudan, where authorities claim to have thwarted coup plots in November and December, and Paraguay, where right-wing legislators pushed leftist President Lugo out of office in June. I didn’t count Sudan because country experts tell me those events were probably just a political ploy President Bashir is using to keep his rivals off balance and not actual coup attempts. I didn’t count Paraguay because President Lugo’s rivals used legal procedures, not force, to oust him in a rushed impeachment. I’m pretty confident that neither of those cases counts as a coup attempt as defined here, but for the sake of argument, it’s worth seeing how the addition of those cases would affect the accuracy assessments.

  • Sudan ranked 11th in the 2012 forecasts, just behind Mali, so the addition of an event there leaves the accuracy stats essentially unchanged at 0.96 and 0.02, respectively.
  • Paraguay would definitely count as a surprise. It ranked in the 80s in the 2012 forecasts, and counting its June events as a coup would drop the AUC to 0.80 and the Brier score to 0.02.
  • If we count both cases as yeses, we get an AUC of 0.84 and a Brier score of 0.02.

All of those are still pretty respectable numbers for true forecasts of rare political events, even if they’re not quite as good as the initial ones. Whatever the exact ground truth, these statistics give me some confidence that the two-model average I’m using here makes a useful forecasting tool.

So, without further ado, what about 2013? The chart below plots estimated coup risk for the coming year for the 30 countries at greatest risk using essentially the same models I used for 2012. (One of the two models differs slightly from last year’s; I cut out a couple of variables that had little effect on the estimates and are especially hard to update.) I picked the top 30 because it’s roughly equivalent to the top quintile, and my experience working with models like these tells me that the top quintile makes a pretty good break point for distinguishing between countries at high and low risk. If a country doesn’t appear in this chart, that means my models think it’s highly unlikely to suffer a coup attempt in the coming year.

2013 Coup Risk Estimates

2013 Coup Risk Estimates

The broad strokes are very similar to 2012, but I’m also seeing a few changes worth noting.

  • Consistent with 2012, countries from sub-Saharan Africa continue to dominate the high-risk group. Nine of the top 10 and 22 of the top 30 countries come from that part of the world. One of those 22 is South Sudan, which didn’t get a forecast in early 2012 because I didn’t have the requisite data but now makes an ignominious debut at no. 20. Another is Sudan, which, as Armin Rosen discusses, certainly isn’t getting any more stable. Mali and Guinea-Bissau also both stay near the top of the list, thanks in part to the “coup trap” I discussed in another recent post. Meanwhile, I suspect the models are overestimating the risk of a new coup attempt in Niger, which seems to have landed on firmer footing after its “democratizing” coup in February 2010, but that recent history will leave Niger in the statistical high-risk group until at least 2015.
  • More surprising to me, Timor-Leste now lands in the top 10. That’s a change from 2012, but only because the data used to generate the 2012 forecasts did not count the assassination attempts of 2008 as a coup try. The latest version of CSP’s coup list does consider those events to be failed coup attempt. Layered on top of Timor-Leste’s high poverty and hybrid political authority patterns, that recent coup activity greatly increases the country’s estimated risk. If Timor-Leste makes it through 2013 without another coup attempt, though, its estimated risk should drop sharply next year.
  • In Latin America, Haiti and Ecuador both make it into the Top 20. As with Timor-Leste, the changes from 2012 are artifacts of adjustments to the historical data—adding a coup attempt in Ecuador in 2010 and counting Haiti as a partial democracy instead of a state under foreign occupation. Those artifacts mean the change from 2012 isn’t informative, but the presence of those two countries in the top 20 most certainly is.
  • Syria also pops into the high-risk group at no. 25. That’s not an artifact of data revisions; it’s a reflection of the effects of that country’s devastating state collapse and civil war on several of the risk factors for coups.
  • Finally, notable for its absence is Egypt, which ranks 48th on the 2013 list and has been a source of coup rumors throughout its seemingly interminable transitional period. It’s worth noting though, that if you consider SCAF’s ouster of Mubarak in 2011 to be a successful coup (CSP doesn’t), Egypt would make its way into the top 30.

As always, if you’re interested in the details of the modeling, please drop me a line at ulfelder@gmail.com and I’ll try to answer your questions as soon as I can.

Update: After a Washington Post blog mapped my Top 30, I produced a map of my own.

Leave a comment

77 Comments

  1. Ian

     /  December 22, 2012

    Thanks for reviewing last years forecasts and projecting out for the next year. Of the African countries on your list, Tanzania jumps out as the most surprising. It has no history of coups and, while power has rested in the hands of only one party since independence and elections on Zanzibar are always problematic, it is generally viewed as a stable democracy (although Mali is just the most recent reminder that “stable democracy” in Africa means very little.). Any thoughts as to why Tanzania is popping up on you model?

    Reply
    • That one surprises me, too, for the reasons you mention. Basically, it’s a poor country with “mixed” political institutions (per Polity) in a region with a lot of armed conflict. Those structural conditions are associated with a high risk of lots of forms of political instability, including coup attempts. The analogy to Mali is apt, as that’s also why Mali scored pretty high in 2012 in spite of not having any recent history of coup attempts, either.

      Reply
      • Grant

         /  December 23, 2012

        I think I’ve mentioned separatism and how it can spark military coups, which would explain Mali. Does your model examine the type of conflict or does it simply take note that conflicts exist?

  2. Quick addendum on 2012 accuracy: I asked Monty Marshall, master keeper of the CSP list, where he’d seen coup attempts in 2012, and he told me Mali, Guinea-Bissau, and Papua New Guinea (January mutiny). Using those cases as the “ground truth” for scoring the 2012 forecasts, I get an AUC of 0.88 and a Brier score of 0.02. Papua New Guinea ranked 55th on the 2012 list, in the second quintile of countries worldwide, so it would have been a surprise, but not as much as Paraguay would have been.

    Reply
  3. Is whether or not elections are scheduled for the year in question taken into account in either model?

    Reply
    • No. I have an indicator for that, but it doesn’t seem to improve the fit of the models. Still, I think your hunch is a good one and suspect that it would be useful for some subset of cases. Maybe I’ll try interacting it with a categorical measure of regime type next time around…

      Reply
  4. Scott

     /  January 8, 2013

    Just wanted to give you my thanks for providing a stellar baseline for further analysis. I always look forward to your posts, and have you at the top of my daily read sites. Cheers!

    Reply
  5. Just curiosity. Who are use this forecasts? Can be arms dealers?

    Reply
    • Well, the Top 30 from the 2013 forecasts are posted on a publicly visible web site, so anyone with an internet connection can use them however they like. At this point, no one else has seen the complete set of 2013 forecasts, which weren’t produced for, or distributed to, any paying clients. My main goals in producing and posting them are 1) to demonstrate that statistical forecasts of rare political events are informative and 2) to show that I can produce good ones.

      Reply
      • Thanks for your explanations. It is very informative for me. Coup forecasts is interesting area to study, and congrats for it. I’ll follow up you from here and twitter. Regards, Ibrahim

  6. It seems that there are a coup attempt in Eritrea :) | BBC News – Eritrea: ‘Troops deployed’ in Asmara http://bbc.in/VKo0W4

    Reply
  7. RAKIB

     /  January 23, 2013

    Surprisingly though the name of PAKISTAN is totally absent bt all count!!!!!!!

    Reply
  8. Your model is flawed if Timor-Leste is in the top 10 countries at risk of a coup, especially if you’re saying there’s a higher risk this year than last year. It just doesn’t work out if you look away from the data and at the country itself.

    Also, the assassination attempts of 2008 were not coup attempts.

    Reply
    • “The model is wrong” is one way to look at it, but the fact that these models have worked pretty well in the past suggests otherwise.

      One way to use forecasting models like this is to look at a country like Timore-Leste and say, “Okay, it looks an awful lot like other countries that have been coup prone in the past—what is there about Timor-Leste that the models are missing?” There very well may be some relevant features that a better set of models would capture, and if you have ideas about what those features are, I’d be glad to hear them.

      Another way to go would be to revise the forecast with that indicator for recent coup activity set to 0 instead of 1, to capture your point about the nature of the events in 2008. If you do that, its 2013 forecast drops markedly, probably pulling it down into the moderate risk group at least. If you were sure of that judgment, you could use the revised forecast. If you weren’t sure, you could average the two, possibly even weighting them by the degree of your confidence in that assertion about the 2008 events.

      Reply
  9. Yusuf Jazairi

     /  January 25, 2013

    I always wonder why Algeria does not figure in the statistical studies on military coups. Algeria has had 4 successful coups (September 1962, June 1965, January 1992 and September 1998) and two failed coup attempts (December 1967 and September 1997). Algeria has had more coups than some of the countries listed above. It has one of the most militarised regimes in Africa and the Arab world : look at SIPRI’s data on military spending, at its military budget relative to the overall government budgets, at the ratio of army+police per inhabitant, the genocidal campaign it has pursued against its political opposition in the 90s (see An Inquiry into Algerian massacres foreworded by Noam Chomsky). The army is not only more powerful than the civilian administration which it controls, but it is also plagued by factional politics – along oil-rent seeking, ideological, generational and corps fault-lines.
    It is this kind of omissions which make me wonder about the extent to which one should take the explanatory/predictive claims of these studies seriously.

    Reply
    • FWIW, Algeria ranks 70th in 2013, just outside the moderate risk group. The fact that it hasn’t had a coup in the past 15 years suggests that’s not crazy, but it sounds like you believe the models are missing something about the militarization of politics there that increase its risk. It would be great to be able to capture more information about military budgets and structures in these models, something I haven’t been able to do so far because of the poor quality of cross-national data on those topics, especially as you go further back in time (which we need to do to train the models).

      Reply
  10. cambo expat

     /  January 25, 2013

    Your 13th choice is so ridiculous and shows no understanding of Cambodian politics, it leaves me to guess this list is pure nonsense from someone who has no idea what is really going on in any of the countries.

    Reply
    • Point taken. But just to be clear, I didn’t “choose” any of the rankings. I chose the models, which generate predicted probabilities, which lead to the rankings. Those predicted probabilities have proved to be pretty well calibrated in the past, and that’s what leads me to believe the forecasts are useful. I certainly don’t claim more than superficial knowledge of the politics of most countries in the world, but the data the models use seem to capture features of those countries’ politics and societies that are reasonably predictive.

      Reply
      • cambo expat

         /  January 25, 2013

        Relying on a model and not knowing anything about the situation in a country seems like pure nonsense. Can your model tell you who will launch a coup? Does your model tell you who owns the guns in the country? There is one country within the ASEAN block where a coup is possible in the next year or 3 and it doesn’t even enter your list. I won’t name it, but look at the last letter of the clause.

      • I think you’re missing my point: the inputs to the models are information “about the situation in a country.” All the models do is select which bits of information to focus on. The selection of those bits, in turn, is based on their demonstrated ability to help distinguish countries where coup attempts have happened in the past from ones where they haven’t.

        Models like these are gross simplifications, of course, but the proof of the pudding is in the eating. I’m sure this task can be done better than I’m doing it here, and I’m always looking to improve the work, but the fact that these models can’t answer every question about a potential coup and don’t fit your prior beliefs about relative coup risk does not invalidate them. Having a bunch of coup attempts occur in countries with relatively low scores would, and we’ll just have to see how that goes.

  11. Vox

     /  January 25, 2013

    How on earth do you have Cambodia in there at 13? I burst out laughing when I saw that. It’s one of the most stable countries in SEAsia with the same Government in power for over fifteen years. Intrigued to know how you feel it ranks a 13 ranking.

    Reply
  12. Shane

     /  January 25, 2013

    The models chosen are clearly too heavily weighted on medium term history in the case of Cambodia (and probably other countries with very turbulent medium term history). Any model which gives any significant weight above a ten year history is likely to give some flawed results. Perhaps you could look at that for future years. Interesting exercise all the same.

    Reply
  13. cambo expat

     /  January 25, 2013

    I am not missing your point. I see a population happily going around buying the latest mobile phones, TVs, motorbikes and cars. I see a population that will re-elect the present government later this year. There is a generation happy that they do do have to fight a war, and the parents are happy their kids don’t. There maybe some problems here but your scaremongering does more to damage a country than anything else.

    Btw sit round in Cambodia and watch and learn and do not rely on statistics. Answer my points.

    Reply
  14. Franck

     /  January 26, 2013

    Very interesting, thanks for publishing this; I just stumbled in from an outside article about your predictions. I am intrigued by Rwanda’s inclusion. I’m here right now for work, and the possibility of a coup is definitely not something I’ve heard anyone mention, quite the contrary. That said, it is a one-party state with a ruler who has been in power nearly two decades, which I think usually don’t bode well. I’m looking forward to reading more about the model, and the situation here, to learn more.

    Reply
    • Thanks for reading and commenting.

      Do bear in mind that the most likely outcome in almost every year for all countries, including the “highest risk” ones, is no coup attempts. You could do a great job forecasting coups by predicting that none will ever happen, and you’d be right about 95% of the time.

      My goal when modeling events as rare and hard to predict as these is to provide some reliable hints about where the occasional events that do occur are most likely to pop up. Given the nature of the processes that produce them and the often-poor quality of the data we can use to forecast them, it’s just not realistic (yet? to make forecasts sharper than that. So seeing the country on this list doesn’t mean the data suggest that a coup is likely there, just that it’s a bit more likely there than in most places.

      Reply
      • cambo expat

         /  January 26, 2013

        And include a system that looks at the facts of the country before including it in a list? If you did that, I think you would get less negative points. You seem to be relying too much on data rather than facts on the ground, leading to some major anomalies in your system.

  15. Reblogged this on Ibrahim ERTURK.

    Reply
  16. As an aspiring international economist I truly appreciate this analysis & would love to study these models. As a current Peace Corps Volunteer in Madagascar I’m not surprised at the statistical likelyhood of a coup. Madagascar is one of the few countries on your list that the Peace Corps still has a presence and for my sake & the Malagasy people I hope there can be a peaceful transfer of power through the upcoming elections.

    Reply
    • Thanks! And just to be clear, the modeling suggests that Madagascar is unlikely to have any coup attempts this year. It’s just that coup attempts are more likely to happen there than in most countries. Put another way, the absolute risk of a coup is low, but the relative risk is high. I hope that makes sense.

      Reply
  17. Reyes

     /  January 31, 2013

    so Afghanistan and Syria are behind Timor Leste ehh?? this is simply ridiculous… if you expect there will be Coup in East Timor in 2013 then i guess you are from another world…

    Reply
    • I don’t expect there’ll be a coup in Timor-Leste in 2013. As the text notes, the estimated risk in any one country is low, even in the “highest risk” cases. It’s just that, if there are coup attempts in 2013—and there probably will be a few of them—I do expect they’ll happen in countries in that “highest risk” group.

      As you can see, that distinction is really important. If you have ideas on how I can communicate it more clearly, I am all ears.

      Reply
  18. Solofo

     /  February 5, 2013

    Why do you think Madagascar is among the top 10, while elections will be held this year in this country?

    Reply
    • It’s not so much an “I think” thing. The forecasts are an average of estimates from statistical models that were trained on historical data from large numbers of cases that did and didn’t experience coup attempts. Based on theory and my knowledge of the data, I choose which variables to include and what type of models to use (logistic regression here), but the parameter estimates and the resulting forecasts come directly from the data, not me.

      Now, on Madagascar specifically, it ranks so high this year because it hits on all three of the strongest risk factors: relatively high infant mortality, “mixed” political institutions, and a recent history of coup activity. The short life span of its current regime and slow economic growth marginally increase its risk as well.

      Reply
      • Solofo

         /  February 5, 2013

        So, according to you, a continuity of transient current does not favor this risk?

      • Sorry, I don’t understand your question.

      • The 2002 crisis happened because of an election, and the 2009 crisis happened within a few months of a planned election. The upcoming election itself has been repeatedly postponed for various more or less believable reasons.

        If anything, I think this year’s elections actually heightens the risk of a coup.

  19. APB

     /  February 16, 2013

    Interesting and clearly passionate responses from so many inhabitants of the countries which made the list. I find your model fascinating because I lived in Mali at the time of the coup and much as other readers have talked about how normal everything is in their country, everyone–including Malians–were absolutely shocked by the coup there. Even as it was happening no one could believe it. It just goes to show how the trigger does not have to stem from a majority viewpoint–minority factions with guns can do alot of damage. Constrasting the complete shock of the citizenry with what took place with your model which predicted it is fascinating to me, especially with enough of a background in stats to understand the inherent weaknesses associated with modeling. I am currently residing in Bangladesh, which is on your list this year. I have to say, you could be right again! Stay tuned…

    Reply
  20. G. I. Ara

     /  February 26, 2013

    I notice that ranking of Bangladesh with respect to prospect of coup has diminished in 2013 compared to 2012. Is it because situation has deteriorated more in other countries?

    Reply
    • No, the predicted probability of a coup in Bangladesh actually fell, because it’s now been more than five years since the last coup or coup attempt. I know there have been claims of foiled coup plots more recently, but for something to count as a coup attempt in these models, it has to progress all the way to an open try.

      Thanks to the Center for Systemic Peace, I have data on coup plots, too, but their inclusion doesn’t seem to improve the predictive power of the model. (In future iterations of the underlying analysis, I plan to look harder for conditional effects from coup plots and rumors. It’s clearly a noisy signal, but I’m convinced there is a signal in there somewhere.)

      Reply
  21. Heather

     /  April 5, 2013

    Interesting work! I’d hypothesize that coups are more likely in countries where the military is strong and there is no other institution/organization strong enough to counter them. Have you thought about the following:
    – Military strength: military spending per troop
    Relative to…
    – Strength of other government institutions: Human Development Index
    – Influence of foreign governments/organizations on military: Foreign assistance as percentage of GDP (military assistance or would be better, but is difficult to find) and presence of peacekeeping troops (especially for Africa)

    Reply
    • Getting reliable and reasonably complete data is always the hardest part of this process, and the measures you suggest are no exception. We don’t have reliable data on military spending and size for a lot of countries, especially as you go back farther in time in order to get a sample large enough to estimate models. Ditto for foreign assistance. Coverage is better in the past couple of decades, but there’ve also been fewer coups during that period.

      Reply
  22. Paul Harper

     /  December 29, 2013

    Vox and Cambo Expat – It is still 2013. Do you still think Cambodia is one of the most stable countries in Asia? I guess not all of the Cambodians are, “happily going around buying the latest mobile phones, TVs, motorbikes and cars.” http://www.globalpost.com/dispatch/news/thomson-reuters/131229/thousands-cambodians-rally-demand-pm-steps-down For DartThrowingChimp It is odd that Thailand doesn’t show up with your system. It is something of an anomaly.

    Reply
    • I’d had the same thought about Cambodia; thanks for pointing it out.

      On Thailand, I agree that the modeling seems to understate the risk. It’ll be interesting to see what the rejiggered model for 2014 does with it.

      Reply
  1. To Read « peter tinti
  2. What We Are Reading 28/12/2012 « Do No Harm
  3. 2013年クーデター予報 » 地球寒冷化ブログ
  4. Conflicts to Watch in 2013 (Part I) « The Widening Lens
  5. Comments and Curios « Political Deficit
  6. Taleb’s Black Swans: The End of Foresight? | Red (team) Analysis
  7. Map: The countries most at risk for a coup in 2013
  8. In Eritrea, Whispers Of A Quiet Protest Challenge Years Of Suppressio..n BY Jacey Fortin | January 22 2013 5:46 AM IBTTimes » erigazette.org | erigazette.org
  9. What’s the Likelihood of a Coup in Eritrea? | Ronin Institute
  10. Coup Risk in 2013, Mapped My Way « Dart-Throwing Chimp
  11. End of Year Predictions: the 2013 Yield | Red (team) Analysis
  12. Conflict and peace-related links of the day « Rachel Strohm
  13. Les 30 coups d’Etat possibles en 2013 | Dreuz.info
  14. Spending in Bangladesh: The most bucks for the biggest bang | BD Watch 2013
  15. Two Forecasting Lessons from a Crazy Football Season « Dart-Throwing Chimp
  16. The most bucks for the biggest bang | DesiRepublic
  17. Coup Forecasts for 2013
  18. End of Year Predictions 2012: How Did they Fare? (2) | Red (team) Analysis
  19. Mali puts coup forecasting in the spotlight – LSMedia
  20. Golden Thread: Why there is no contradiction between helping the poorest and our national interest | Platform 10
  21. Comments & Curios IV | Political Deficit
  22. Forecasting Round-Up No. 4 | Dart-Throwing Chimp
  23. Common Predictive Analytics Screw-Ups, Social-Science Edition | Dart-Throwing Chimp
  24. Bangladesh as Archetype of Contemporary Political Development | Dart-Throwing Chimp
  25. Using Data To Predict Civil Unrest In Africa | AFKInsider
  26. Follow-Up on Bangladesh | Dart-Throwing Chimp
  27. Most Popular Posts of 2013 | Dart-Throwing Chimp
  28. A Research Note on Updating Coup Forecasts | Dart-Throwing Chimp
  29. Konfliktbevakning 2014: Bangladesh | Johan Larnefeldt
  30. Coup Forecasts for 2014 | Dart-Throwing Chimp
  31. 전세계의 최신 영어뉴스 듣기 - 보이스뉴스 잉글리쉬
  32. Was there a coup in Iraq? What we know and what we don’t know | The Yam Times

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 7,816 other followers

%d bloggers like this: