This blog’s gotten a lot more traffic than usual since yesterday, when Max Fisher of the Washington Post called out my 2013 coup forecasts in a post on WorldViews.
I’m grateful for the attention Max has drawn to my work, but if it had been up to me, I would have done the mapping a little differently. As I said to Max in an email from which he later excerpted, the forecasts simply aren’t sharp enough to parse the world as finely as their map did. Our theories of what causes coup attempts are too fuzzy and our measures of the things in those theories are too spotty to estimate the probability of these rare events with that much precision.
But, hey, I’m a data guy. I don’t have to stick to grumbling about the Post‘s map; I can make my own! So…
The map below sorts the countries of the world into three groups based on their relative coup risk for 2013: highest (red), moderate (orange), and lowest (beige). I emphasize “relative” because coup attempts are very rare, so the estimated risk of coup attempts in any given country in any single year is pretty small. For example, Guinea-Bissau tops my list for 2013, and the estimated probability of at least one coup attempt occurring there this year is only 25%. Most countries worldwide are under 2%.
Consistent with an emphasis on relative risk, the categories I’ve mapped are based on rank order, not predicted probability. The riskiest fifth of the world (33 countries) makes up the “highest” group, the second fifth the “moderate” group, and the bottom three-fifths the “lowest” group.
This forecasting process doesn’t have enough of track record for me to say exactly how those categories relate to real-world risk, but based on my experience working with similar data and models, I would expect roughly four of every five coup attempts to occur in countries identified here as high risk, and the occasional “miss” to come from the moderate-risk set. Only very rarely should coup attempts come from the 100 or so countries in the low-risk group.
Dart-Throwing Chimp 
lissnup
/ January 23, 2013Fascinating stuff! If you evaluate your probabilities by counting failed attempts as well as successful coups, then I think you earned points with Mauritania last year, where the president’s alleged friendly-fire shooting injury in October 2012 has never been fully accepted or satisfactorily explained.
hearabout
/ February 3, 2013Hello! First, I really enjoy your blog and the data based policy forecasts. I also read through the comments on your original post about 2013. However, I still don’t quite get what data you influences your modelling? As a previous visitor pointed out it is very surprising to find Tanzania in the red. Tanzania is characterised by an extremely peaceful history, it has very little internal clashes based on ethnicity. There are growing protests sparked by young men, mostly muslim, but those are sparse and rooted in general economic discontent rather than religiously motivated. I would say that the only internal conflict potential lies in the political unrest on Zanzibar. Mainland Tanzania has a stable political system and public debates on the upcoming elections or the new constitution are very moderate and peaceful. If all those factors are not considered in your model, what is?
dartthrowingchimp
/ February 4, 2013The variables used in the algorithm that produces these risk estimates are enumerated in my post on the 2012 forecasts (here).
Regarding Tanzania in particular, the algorithm is identifying it as a higher-risk case mostly because of its relatively high infant mortality rate and its “mixed” political institutions. The presence of ongoing armed conflicts in its geographic region also adds to the score, but only very marginally.
A lot of people are surprised to see it rank so high on this list for the reasons you note. I think there are two ways to interpret that. One, it’s quite possible that your skepticism is warranted and the models’ gross sorting is missing some important features of Tanzanian politics that render coup attempts even more improbable than this this forecast indicates. Two, it’s also possible that, as happened with Mali before 2012, the standing view of Tanzania is overstating the stability of its politics and there really is some risk there that the conventional wisdom is overlooking.
I can’t say which is more correct, of course. What I can say is that I think it makes sense for people interested in using forecasts like these to think about how this forecast might warrant an adjustment of their prior belief—or not, if that’s how it turns out—and Bayes’ theorem is a very useful tool for doing that in a structured way.
More generally, I’d say that this discussion we’re having right now is exactly the point of forecasts like these. It’s not very interesting or useful to say that Guinea-Bissau is at relatively high risk of a coup in 2013 and the U.K. is not. What’s more useful is to use patterns in data to identify and spur conversation about some surprising cases, like Tanzania. That conversation will often lead us right back to our prior beliefs, but sometimes it will help us revise those beliefs in a way that usefully reduces surprise.
hearabout
/ February 4, 2013Thank you for your detailed answer! I completely agree with you on the point that obviously statistics have to be interpreted within the framework of reality, but oftentimes reality is a quite subjective affair and “hard” numerical data helps us to adjust our impressions and believes. Re Tanzania, I just read a statistic that it has the highest rates in muggins in the region and when I think back on the “safety” I felt in Dar es Salaam it might be more of a retrospective imagination… I have been mugged twice in three months…
So the best possible outcome of your study could be that some Tanzanian or any other government representatives that are in the red look at your studies and put their thinking caps on to find ways to strengthen their systems…. Keep the good work going! 🙂
William Church (@CS3Corg)
/ January 25, 2014I guess I have a strange view of things since I spent my entire career in the field. But in my opinion, I think we need to look at the word coup (sorry if I missed a definition on your site) more closely before we can predict. First, lets take the current action South Sudan a country that I do not believe appeared on your list for 2012. SoSu is a classic military/political coup. Not popular in nature per se though supported by popular groups. One group with weapons disputes another political group with weapons. Very few things on your list of factors has anything at all to do with that coup. Now, lets take Egypt (A country I have lived in) which also did not appear on any of your lists. Now that is an social unrest coup in the first instance and a military coup in the second. It could be influenced by personal social factors. So I think we maybe need to talk about change of government versus coup. Lets take Rwanda (a country I have lived in) you can add all the data points you want but the odds of a coup are almost zero to none. Why? Unified military/police power, and unified political (yes obtained in a special way) power. Your forecast missed the Ukraine completely as well as Thailand. For good reason they are developed (developing countries) with very few of the “data points” you needed to forecast. Conversely, the prediction of Sudan is close to pretty far fetched unless my friend Bashir loses his grip on the military then pretty darn likely. Anyway your work is interesting but nothing beats HUMINT (with strong data).
dartthrowingchimp
/ January 25, 2014Let me start by saying that this is not an either-or question. I’m with Nate Silver here: forecasters should be information omnivores. That said, I think there are some unique benefits that statistical forecasts bring to helping assess risks or prospects of hard-to-foresee events like coups.
First, there’s the efficiency. It took one (experienced) person working off and on over a few days to assemble and analyze the data to produce these forecasts, which cover the whole world and, past performance suggests, should be fairly accurate. That’s orders of magnitude cheaper than any HUMINT program required to produce a similar product which may or may not be more accurate.
Second, statistical models can complement that HUMINT effort by helping to focus collection. If your a program director with a limited budget trying to decide where to target your efforts to discern risks, forecasts like these can help you make better decisions.
Third, statistical forecasts can provide a useful benchmark for comparing the warnings your HUMINT produces. Imagine you’re NSC director or CEO or editor and you’ve got 10 different analysts or writers telling you that their country of concern is at risk of coup. How do you compare those warnings? Forecasts from a model like this one give you a set of priors that can help you sort that information more carefully.
William Church (@CS3Corg)
/ January 25, 2014Agree, Note I said with HUMINT with strong data. Sorry, if you saw my comment as disrespectful in any way; however, I think we both might admit there are some flaws with the predictive nature of model. Hardly, am I proposing all HUMINT but your model, please correct me if wrong, appears to be largely data driven. Regardless, there is plenty room in the world for different approaches and a combination of approaches. But nothing replaces field experience.