Why My Coup Risk Models Don’t Include Any Measures of National Militaries

For the past several years (herehere, here, and here), I’ve used statistical models estimated from country-year data to produce assessments of coup risk in countries worldwide. I rejigger the models a bit each time, but none of the models I’ve used so far has included specific features of countries’ militaries.

That omission strikes a lot of people as a curious one. When I shared this year’s assessments with the Conflict Research Group on Facebook, one group member posted this comment:

Why do none of the covariates feature any data on militaries? Seeing as militaries are the ones who stage the coups, any sort of predictive model that doesn’t account for the militaries themselves would seem incomplete.

I agree in principle. It’s the practical side that gets in the way. I don’t include features of national militaries in the models because I don’t have reliable measures of them with the coverage I need for this task.

To train and then apply these predictive models, I need fairly complete time series for all or nearly all countries of the world that extend back to at least the 1980s and have been updated recently enough to give me a useful input for the current assessment (see here for more on why that’s true). I looked again early this month and still can’t find anything like that on even the big stuff, like military budgets, size, and force structures. There are some series on this topic in the World Bank’s World Development Indicators (WDI) data set, but those series have a lot of gaps, and the presence of those gaps is correlated with other features of the models (e.g., regime type). Ditto for SIPRI. And, of course, those aren’t even the most interesting features for coup risk, like whether or not military promotions favor certain groups over others, or if there is a capable and purportedly loyal presidential guard.

But don’t take my word for it. Here’s what the Correlates of War Project says in the documentation for Version 4.0 of its widely-used data set (PDF) about its measure of military expenditures, one of two features of national militaries it tries to cover (the other is total personnel):

It was often difficult to identify and exclude civil expenditures from reported budgets of less developed nations. For many countries, including some major powers, published military budgets are a catch-all category for a variety of developmental and administrative expenses—public works, colonial administration, development of the merchant marine, construction, and improvement of harbor and navigational facilities, transportation of civilian personnel, and the delivery of mail—of dubious military relevance. Except when we were able to obtain finance ministry reports, it is impossible to make detailed breakdowns. Even when such reports were available, it proved difficult to delineate “purely” military outlays. For example, consider the case in which the military builds a road that facilitates troops movements, but which is used primarily by civilians. A related problem concerns those instances in which the reported military budget does not reflect all of the resources devoted to that sector. This usually happens when a nation tries to hide such expenditures from scrutiny; for instance, most Western scholars and military experts agree that officially reported post-1945 Soviet-bloc totals are unrealistically low, although they disagree on the appropriate adjustments.

And that’s just the part of the “Problems and Possible Errors” section about observing the numerator in a calculation that also requires a complicated denominator. And that’s for what is—in principle, at least—one of the most observable features of a country’s civil-military relations.

Okay, now let’s assume that problem magically disappears, and COW’s has nearly-complete and reliable data on military expenditures. Now we want to use models trained on those data to estimate coup risk for 2015. Whoops: COW only runs through 2010! The World Bank and SIPRI get closer to the current year—observations through 2013 are available now—but there are missing values for lots of countries, and that missingness is caused by other predictors of coup risk, such as national wealth, armed conflict, and political regime type. For example, WDI has no data on military expenditures for Eritrea and North Korea ever, and the series for Central African Republic is patchy throughout and ends in 2010. If I wanted to include military expenditures in my predictive models, I could use multiple imputation to deal with these gaps in the training phase, but then how would I generate current forecasts for these important cases? I could make guesses, but how accurate could those guesses be for a case like Eritrea or North Korea, and then am I adding signal or noise to the resulting forecasts?

Of course, one of the luxuries of applied forecasting is that the models we use can lack important features and still “work.” I don’t need the model to be complete and its parameters to be true for the forecasts to be accurate enough to be useful. Still, I’ll admit that, as a social scientist by training, I find it frustrating to have to set aside so many intriguing ideas because we simply don’t have the data to try them.

Leave a comment

5 Comments

  1. We used (imputed) military expenditures from the World Bank in one of the models that went into our ensemble forecast of irregular leadership changes, and also considered the personnel size of the military, but neither was very helpful in the end. Granted, we use monthly data and our outcome of interest is broader, but since 30-40% of our cases consist of coups, maybe there’s something relevant.

    In addition to the data problems you mention, I think another challenge is that at least sometimes concepts/variables one might think are theoretically important and which might have shown statistically significant relationships in previous research turn out to not be good predictors. The more common assumption still seems to be that low p = good predictor, or at least that the onus is to show otherwise.

    Reply
    • Andy, on your point about statistical significance vs. predictive power, amen.

      On measuring militaries, when it came time to generate forecasts, how did you all handle cases with no values ever (e.g., North Korea and Eritrea) or no recent values (e.g., CAR)? I can think of some strategies that would let me get to a forecast; I just worry that those inputs would be so uncertain for crucial cases like the ones I just mentioned that the additional noise in the final prediction would outweigh the benefits of having the thing in the equation.

      Reply
  2. Cyrus

     /  January 27, 2015

    I’m working on a coup forecasting project myself and have come up against this same issue. I found the SIPRI data to be more reliable based on a rough qualitative assessment but overall i think SIPRI~COWmilex yields an r2 of .92ish (if i remember correctly). Because SIPRI is limited to post1988 (if my memory serves me correct) i just used the COW data to obtain predicted values for the SIPRI data running from ’88 back. Of course this doesn’t solve the problem of nonrandom missing data and imputation as you suggested may create new problems even as it solves old ones -inference just becomes prohibitively difficult when including military expenditure.

    Also, im not entirely convinced any of these measures of the military even have a significant impact on coup risk. Restricting my model’s sample to 1992-2010 and using multiple imputation i found no measures of military expenditure (lack of growth in military spending as a binary variable or even fraction, aggregate military spending, military spending as a proportion of national budget, military spending per capita, etc.) that were at all significantly correlated with coup risk and the signs of the coefficients were changing with my changing operationalizations of the concept -perhaps theres too much noise but maybe theres simply no link. As far back as 1982 Gary Zuk and William Thompson argued against the concept of corporate self interest on the part of the military; they found no link between military spending and coups. Maybe there isn’t one?

    Finally, in regards to the COW codebook’s statement of assessing military expenditures. Frankly I’m not convinced a ‘good’ measure of military expenditure should, as you suggest, distinguish between ‘public works, colonial administration, development of the merchant marine, construction, and improvement of harbor and navigational facilities, transportation of civilian personnel, and the delivery of mail’ and more militarily ‘relevant’ activities. The reason being is that if we accept the theory of military corporate self interest (the notion that military officers act according to the interest of the institution that they represent) then what that money is used for is a lot less important than through whose hands that money travels. In Iran, for example, the revolutionary guard is tasked with the biggest and perhaps most lucrative public works projects despite the fact that a separate public works department exists within each province. High ranking officers in the revolutionary guard use these no bid public works contracts to amass large budgets that they can use to skim off the top. I’m not so sure it matters what the money is used for as long as the military gets to allocate its use. In this sense it seems appropriate to include any and all budgetary considerations made under the label ‘military’.

    Reply
    • Cyrus, thanks very much for sharing all of that. Andy Beger reports a similar finding on military expenditures (see comment above), so I’m inclined to agree with you, that there doesn’t seem to be much of a link there. Not with the (noisy) available data, anyway. And on COW’s data handbook, I didn’t mean to endorse every part of that quoted statement. I was just trying to convey how much harder it is to measure this thing than most people probably imagine (“Just find the right line item in the annual budget table and, voila!”).

      Reply
  1. To Realize the QDDR’s Early-Warning Goal, Invest in Data-Making | Dart-Throwing Chimp

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: