Mining Texts to Generate Fuzzy Measures of Political Regime Type at Low Cost

Political scientists use the term “regime type” to refer to the formal and informal structure of a country’s government. Of course, “government” entails a lot of things, so discussions of regime type focus more specifically on how rulers are selected and how their authority is organized and exercised. The chief distinction in contemporary work on regime type is between democracies and non-democracies, but there’s some really good work on variations of non-democracy as well (see here and here, for example).

Unfortunately, measuring regime type is hard, and conventional measures of regime type suffer from one or two crucial drawbacks.

First, many of the data sets we have now represent regime types or their components with bivalent categorical measures that sweep meaningful uncertainty under the rug. Specific countries at specific times are identified as fitting into one and only one category, even when researchers knowledgeable about those cases might be unsure or disagree about where they belong. For example, all of the data sets that distinguish categorically between democracies and non-democracies—like this one, this one, and this one—agree that Norway is the former and Saudi Arabia the latter, but they sometimes diverge on the classification of countries like Russia, Venezuela, and Pakistan, and rightly so.

Importantly, the degree of our uncertainty about where a case belongs may itself be correlated with many of the things that researchers use data on regime type to study. As a result, findings and forecasts derived from those data are likely to be sensitive to those bivalent calls in ways that are hard to understand when that uncertainty is ignored. In principle, it should be possible to make that uncertainty explicit by reporting the probability that a case belongs in a specific set instead of making a crisp yes/no decision, but that’s not what most of the data sets we have now do.

Second, virtually all of the existing measures are expensive to produce. These data sets are coded either by hand or through expert surveys, and routinely covering the world this way takes a lot of time and resources. (I say this from knowledge of the budgets for the production of some of these data sets, and from personal experience.) Partly because these data are so costly to make, many of these measures aren’t regularly updated. And, if the data aren’t regularly updated, we can’t use them to generate the real-time forecasts that offer the toughest test of our theories and are of practical value to some audiences.

As part of the NSF-funded MADCOW project*, Michael D. (Mike) Ward, Philip Schrodt, and I are exploring ways to use text mining and machine learning to generate measures of regime type that are fuzzier in a good way from a process that is mostly automated. These measures would explicitly represent uncertainty about where specific cases belong by reporting the probability that a certain case fits a certain regime type instead of forcing an either/or decision. Because the process of generating these measures would be mostly automated, they would be much cheaper to produce than the hand-coded or survey-based data sets we use now, and they could be updated in near-real time as relevant texts become available.

At this week’s annual meeting of the American Political Science Association, I’ll be presenting a paper—co-authored with Mike and Shahryar Minhas of Duke University’s WardLab—that describes preliminary results from this endeavor. Shahryar, Mike, and I started by selecting a corpus of familiar and well-structured texts describing politics and human-rights practices each year in all countries worldwide: the U.S. State Department’s Country Reports on Human Rights Practices, and Freedom House’s Freedom in the World. After pre-processing those texts in a few conventional ways, we dumped the two reports for each country-year into a single bag of words and used text mining to extract features from those bags in the form of vectorized tokens that may be grossly described as word counts. (See this recent post for some things I learned from that process.) Next, we used those vectorized tokens as inputs to a series of binary classification models representing a few different ideal-typical regime types as observed in few widely used, human-coded data sets. Finally, we applied those classification models to a test set of country-years held out at the start to assess the models’ ability to classify regime types in cases they had not previously “seen.” The picture below illustrates the process and shows how we hope eventually to develop models that can be applied to recent documents to generate new regime data in near-real time.

Overview of MADCOW Regime Classification Process

Overview of MADCOW Regime Classification Process

Our initial results demonstrate that this strategy can work. Our classifiers perform well out of sample, achieving high or very high precision and recall scores in cross-validation on all four of the regime types we have tried to measure so far: democracy, monarchy, military rule, and one-party rule. The separation plots below are based on out-of-sample results from support vector machines trained on data from the 1990s and most of the 2000s and then applied to new data from the most recent few years available. When a classifier works perfectly, all of the red bars in the separation plot will appear to the right of all of the pink bars, and the black line denoting the probability of a “yes” case will jump from 0 to 1 at the point of separation. These classifiers aren’t perfect, but they seem to be working very well.

 

prelim.democracy.svm.sepplot

prelim.military.svm.sepplot

prelim.monarchy.svm.sepplot

prelim.oneparty.svm.sepplot

Of course, what most of us want to do when we find a new data set is to see how it characterizes cases we know. We can do that here with heat maps of the confidence scores from the support vector machines. The maps below show the values from the most recent year available for two of the four regime types: 2012 for democracy and 2010 for military rule. These SVM confidence scores indicate the distance and direction of each case from the hyperplane used to classify the set of observations into 0s and 1s. The probabilities used in the separation plots are derived from them, but we choose to map the raw confidence scores because they exhibit more variance than the probabilities and are therefore easier to visualize in this form.

prelim.democracy.svmcomf.worldmap.2012

prelim.military.svmcomf.worldmap.2010

 

On the whole, cases fall out as we would expect them to. The democracy classifier confidently identifies Western Europe, Canada, Australia, and New Zealand as democracies; shows interesting variations in Eastern Europe and Latin America; and confidently identifies nearly all of the rest of the world as non-democracies (defined for this task as a Polity score of 10). Meanwhile, the military rule classifier sees Myanmar, Pakistan, and (more surprisingly) Algeria as likely examples in 2010, and is less certain about the absence of military rule in several West African and Middle Eastern countries than in the rest of the world.

These preliminary results demonstrate that it is possible to generate probabilistic measures of regime type from publicly available texts at relatively low cost. That does not mean we’re fully satisfied with the output and ready to move to routine data production, however. For now, we’re looking at a couple of ways to improve the process.

First, the texts included in the relatively small corpus we have assembled so far only cover a narrow set of human-rights practices and political procedures. In future iterations, we plan to expand the corpus to include annual or occasional reports that discuss a broader range of features in each country’s national politics. Eventually, we hope to add news stories to the mix. If we can develop models that perform well on an amalgamation of occasional reports and news stories, we will be able to implement this process in near-real time, constantly updating probabilistic measures of regime type for all countries of the world at very low cost.

Second, the stringent criteria we used to observe each regime type in constructing the binary indicators on which the classifiers are trained also appear to be shaping the results in undesirable ways. We started this project with a belief that membership in these regime categories is inherently fuzzy, and we are trying to build a process that uses text mining to estimate degrees of membership in those fuzzy sets. If set membership is inherently ambiguous in a fair number of cases, then our approximation of a membership function should be bimodal, but not too neatly so. Most cases most of the time can be placed confidently at one end of the range of degrees of membership or the other, but there is considerable uncertainty at any moment in time about a non-trivial number of cases, and our estimates should reflect that fact.

If that’s right, then our initial estimates are probably too tidy, and we suspect that the stringent operationalization of each regime type in the training data is partly to blame. In future iterations, we plan to experiment with less stringent criteria—for example, by identifying a case as military rule if any of our sources tags it as such. With help from Sean J. Taylor, we’re also looking at ways we might use Bayesian measurement error models to derive fuzzy measures of regime type from multiple categorical data sets, and then use that fuzzy measure as the target in our machine-learning process.

So, stay tuned for more, and if you’ll be at APSA this week, please come to our Friday-morning panel and let us know what you think.

* NSF Award 1259190, Collaborative Research: Automated Real-time Production of Political Indicators

China and Russia and What Could Have Happened

Twenty five years ago, I was strolling down Leningrad’s main drag, Nevsky Prospekt, with a clutch of other American undergraduates who had recently arrived for two months of intensive language study when Professor Edna Andrews dashed up to us with the news. “They’re shooting them,” she said (or something like it—who can trust a 25-year-old memory of a speech fragment?) with obvious agitation. “They’re shooting the students in Tiananmen Square!”

Had Edna not given us that news, we probably wouldn’t have heard it, or at least not until we got home. In 1989, glasnost’ had already come to the USSR, but that didn’t mean speech was free. State newspapers were still the only ones around, at least for those of us without connections to the world of samizdat. Some of those newspapers were more informative than others, but the limits of political conversation were still clearly proscribed. The Internet didn’t exist, and international calls could only be made by appointment from state-run locations with plastic phones in cubicle-like spaces and who-knows who listening while you talked. Trustworthy information still only trickled through a public sphere mostly bifurcated between propaganda and silence.

What’s striking to me in retrospect is how differently things could have turned out in both countries. When she gave us the news about Tiananmen, Edna was surely agitated because it involved students like the ones she taught being slaughtered. I suspect she was also distressed, though, because at the time it was still easy to imagine something similar happening in the USSR, perhaps even to people she knew personally.

In 1989, politics had already started to move in the Soviet Union, but neither democratization nor disintegration was a foregone conclusion. That spring, citizens had picked delegates to the inaugural session of the Congress of People’s Deputies in elections that were, at the time, the freest the USSR had ever held. The new Congress’ sessions were shown on live television, and their content was stunning. “Deputies from around the country railed against every scandal and shortcoming of the Soviet system that could be identified,” Thomas Skallerup and James P. Nichol describe in their chapter for the Library of Congress’ Russia country study. “Speakers spared neither Gorbachev, the KGB, nor the military.”

But the outspokenness of those reformist deputies belied their formal power. More than 80 percent of the Congress’ deputies were Communist Party members, and the new legislative body the deputies elected that summer, the Supreme Soviet of the USSR, was stuffed with “old-style party apparatchiks.” Two years later, reactionaries inside the government mounted a coup attempt in which President Gorbachev was arrested and detained for a few days and tanks were deployed on the streets of Moscow.

Tank near Red Square on 19 August 1991. © Anatoly Sapronyenkov/AFP/Getty Images

That August Putsch looks a bit clowny with hindsight, but it didn’t have to fail. Likewise, the brutal suppression of China’s 1989 uprising didn’t have to happen, or to succeed when it did. In a story published this week in the New York Times, Andrew Jacobs and Chris Buckley describe the uncertainty of Chinese policy toward the uprising and the disunity of the armed forces tasked with executing it—and, eventually, the protesters in Tiananmen Square.

“At the time,” Jacobs and Buckley write, “few in the military wanted to take direct responsibility for the decision to fire on civilians. Even as troops pressed into Beijing, they were given vague, confusing instructions about what to do, and some commanders sought reassurances that they would not be required to shoot.” Seven senior commanders signed a petition calling on political leaders to withdraw the troops. Those leaders responded by disconnecting many of the special phones those commanders used to communicate with each other. When troops were finally given orders to retake the square “at any cost,” some commanders ignored them. At least one pretended that his battalion’s radio had malfunctioned.

As Erica Chenoweth and Maria Stephan show in their study of civil resistance, nonviolent uprisings are much more likely to succeed when they prompt defections by security forces. The Tiananmen uprising was crushed, but history could have slipped in many other directions. And it still can.

Realists Versus Russianists on Ukraine

In my neck of the intellectual woods, views on the causes of Russia’s recent aggression in Ukraine split into two camps, which I’ll call the Realists and the Russianists. Some people straddle the two groups, and there are other streams of thought on this issue, but mostly the conversation seems to break into competing claims along the following lines.

For their part, the Realists explain Russia’s behavior as a product of the insecurity endemic to the international system. Realism sees world politics as fundamentally anarchic, and the absence of a trusted protector encourages states to husband and grow their power as the only reliable means of protecting their survival and autonomy. This insecurity is so compelling that it even tempts states into expansionist actions aimed at preempting future threats, either directly by winning control over the menacing rival or indirectly by increasing their own power. As Arthur Eckstein (2006, p. 21) describes,

In an anarchic state-system where relations are strongly competitive and conflictual, where unresolved disputes, tensions, and frictions accumulate, and where every state must look primarily to its own resources in order to defend itself, it makes sense for every state to take the strongest possible measures to increase its own security. But measures successfully taken to increase one state’s security act simultaneously to undermine the security of every other state. The result is that the level of tension and distrust intensifies all around. Moreover, making a competitor (i.e., a potential adversary) more insecure not only tends to increase the other state’s interest in militarization but also—as Realists judge from conditions in the modern world—the other state’s own interest in territorial expansion.

Seen through this lens, the revolution in Ukraine in February created both a threat and an opportunity for Moscow. After Yanukovich fled his post, Russia was especially eager to secure control over Crimea because it is home to the Black Sea Fleet, but Putin’s government was also motivated to grab at parts of Ukraine by the continuing expansion of a rival power that already abuts its borders. For Realists, it is not coincidental that Russia’s most aggressive gambits in the past decade have come in Georgia and Ukraine, two countries that border Russia and to which NATO and the E.U. have continued to dangle the promise of eventual membership. Nor is it surprising that Russia seized a chunk of Ukraine and continues to destabilize the rest of it when it looked like Kiev was finally poised to take a clear step toward “the West” by signing an E.U. association agreement and rejecting membership in Moscow’s rival project, the Eurasian Customs Union.

So that’s where the Realists seem to be coming from. The Russianists, meanwhile, see Moscow’s aggression in Ukraine almost entirely as the product of a domestic turn toward chauvinism—or, for the more fatalistic, the resurgence of a chauvinism that never really goes away. Some Russianists see the Putin administration’s waxing nationalism as an earnest manifestation of its core values, while others view it more cynically as a ploy to rally domestic support for the government in the face of a sputtering economy and frustration with authoritarian rule. In either case, though, the root cause of Russian aggression is said to lie within its own borders, and NATO and the E.U. are potential partners whom Russia is rebuffing because its own pathologies blind it to the mutual gains that more cooperative relations would produce.

As is often the case, these competing explanations also imply different expectations about the future. Many of the Russianists I follow have reacted to the annexation of Crimea with predictions that a Kremlin crazy (like a fox?) with nationalism will not stop until it is compelled to stop. They now expect the eventual annexation of eastern and possibly southern Ukraine and see Moldova’s Transdniestr region as a probable next target. Some even warn that the presence of large numbers of co-ethnics in Estonia and Latvia will compel Russia to venture there, and they fear that those countries’ NATO membership, and the untested promise of collective defense it represents, will not suffice to deter this ambition. In their view, the logic of Russian domestic politics now dictates an expansionist line that can only be broken by crippling punishment and loud threats of harsh retaliation.

By contrast, many of the Realists seem less alarmed about where this crisis goes from here. They believe that the same insecurity which motivated Moscow’s recent aggression will also discourage Putin & co. from pushing much farther against the interests of a powerful rival, especially when Russia’s own hard-power resources are already stretched thin and maybe even shrinking. Instead, they expect Moscow to focus on consolidating its recent gains and to avoid provoking an even costlier response by probing the boundaries of NATO’s resolve.

Whether Moscow holds the current line or moves to annex more territory won’t prove one camp right and the other wrong. A single episode will rarely suffice for that, and either course is at least possible under either framework. What it will do, however, is provide new information on which we can and should update our thinking about which view is more right. If Russia stops at Crimea, it would undercut many Russianists’ claims that Putin & co. are compelled by their chauvinist project to “protect” ethnic kin and to continue regrowing the empire. If Moscow does pushes much further, however—and, especially, if it ventures into the Baltic states—it would be hard to sustain the Realist line that domestic nationalism is largely ephemeral to this process.

Importantly, there’s also a prescriptive side to this debate. If the Russianists are (more) right, then now is the Munich moment, and the only way to halt Moscow’s expansionist drive is to make clear to the Putin government that it will suffer badly for its transgressions. If the Realists are (more) right, however, then the bellicose response some Russianists seek could exacerbate the situation by amplifying the security dilemma that underlies the current crisis.

For what it’s worth, my views on this particular case hew closer to the Realists. Russian domestic politics is surely influencing Moscow’s behavior—how could it not?—but if I had to pick a single class of events to which this episode belongs, I would put it in the bin with the many, many other instances of aggressive self-help in response to the insecurity endemic to an anarchic international system. Observers of world politics have noted this pattern for literally thousands of years. Meanwhile, the nationalist turn we’re seeing in Russian domestic politics could be as much consequence as cause of these international tensions, and the idea of Russia as pathologically expansionist is hard to square with the limited scope of Russian irridentism we’ve actually seen over the past 20 years. Europe and the United States may insist and genuinely believe that they mean Russia no harm, but their active and vocal pursuit of democratic change in Moscow undercuts this claim, at least in the eyes of the people who staff the apparatus that actually makes Russian foreign policy. That pattern doesn’t justify Russian aggression, but I do think it goes a longer way toward explaining it.

Forecasting Coup-ish Events

This is a guest post written by Matt Reichert, Miguel Garces, Quratul-Ann Malik, and Ian Lustick of Lustick Consulting. Any questions about this post, these models, Lustick Consulting, or agent-based modeling can be directed to info@lustickconsulting.com.

For students of civil-military relations, the ouster of Ukrainian ex-President Viktor Yanukovich  is puzzling—it’s just not quite clear what to call it. Unlike the case of Egypt, where the checklist of coup criteria can be marked down with a single swift sentence, classifying the Ukraine case has required much more thought. With parliament as the chief perpetrator, and without the Ukrainian armed forces or military playing a significant role, this coup feels like a fire without the smoke. It is not so much a coup, as coup-ish.

How do we forecast “coup-ish?” We suggest that the Ukraine case typifies the type of forecasting challenge that is best met by conceptual disaggregation. In his treatment of Ukraine, what Ulfelder has captured is the unique analytic role played by what has been termed “diminished subtypes.” According to David Collier and Steven Levitsky (1997), a diminished subtype appends and subtracts one or more criteria from some root concept, making it related to, but not quite a pure specification of, that concept. It is the root concept “with adjectives.” Their example is ‘illiberal democracy,’ which does not meet the minimal requirements of the root concept ‘democracy,’ and includes additional criteria and a smaller number of cases than the root concept democracy. Yet there is still important analytic utility to defining the concept in connection with democracy.

What happened in Ukraine might be classified not as a pure ‘coup,’ yet for Ulfelder, it is clearly close enough that recognizing it as ‘coup-ish’ in some way is analytically useful. Thus, we propose the diminished subtype ‘parliamentary coup.’

For this to hold, the diminished subtype ‘parliamentary coup’ must satisfy most, break at least one, and add at least one criterion to the existing definitional criteria for the root concept “coup.” To understand how this might be accomplished, it is useful to first re-state the three coup definitions used by Ulfelder – one each from the coding rules for the two datasets used to validate his coup forecasts, and one from Ulfelder’s own commentary on Ukraine – and identify where they are congruent and diverge. Following the example of Powell and Thyne, we break down the definitions into three components.

Target

Perpetrator

Tactics

Powell & Thyne
(Ground truth A)
Chief executive Any elite who is part of the state apparatus Illegal; no minimal death criteria
Marshall & Marshall
(Ground truth B)
Executive authority A dissident/opposition faction within the country’s ruling or political elites A forceful seizure
Ulfelder
(Ukraine commentary)
Chief executive Political insiders Do not follow constitutional procedure; involves use or threat of force

There is general agreement here on the target, chief executive, and some agreement on the tactic—it must be extra-legal and in some way forceful. There is disagreement on the perpetrator—whether it must be a state actor, or simply a political insider.

Another way of conceptualizing these definitions and their inter-relationships is that extra-legal seizure of the chief executive office operates as an overarching abstract concept. By specifying the perpetrator, we build the root concept ‘coup:’ an extra-legal seizure of the chief executive office by some state actor. When we adjust the specification of the perpetrator, we get our diminished subtype ‘parliamentary coup:’ an extra-legal seizure of the chief executive office by an unarmed political insider. The diminished subtype shares but also excludes some criteria of the root concept, and both fit comfortably under the over arching abstract concept, as illustrated in the figure below.

coup typology graphic

We included here the subtype ‘military coup’ to distinguish the conventional tactic of specification from the diminished subtype route. While the diminished subtype satisfies some but not all of the criteria of the root concept, the conventional subtype exists entirely within the root concept, and simply drops the level of generality by adding criteria and reducing the number of qualifying cases. Thus, while all military coups are coups, not all coups are military coups – and all parliamentary coups are not quite coups, but still coup-ish.

What does classifying our classification scheme buy us? The added value is more than semantic. How we choose to conceptualize, and being self-aware about those choices, can shape and affect how we analyze, how we model, and also how we forecast. As Giovanni Sartori observed (1970), “concept formation stands prior to quantification.” Here, we will demonstrate how using these different conceptualizations in a model, and taking on the analytic baggage that comes with each, affects not just the forecasts we make, but also the questions we ask about those forecasts.

An agent-based model makes for especially fertile ground for this type of introspective test, for two reasons. First, it is a causal model, which means that the individual chains of implications for each type of concept are available for interrogation. Second, the animating principle behind the agent-based modeling approach is complexity, which incorporates the overlapping chains of causation and nth order effects produced by thousands of individually specified interacting agents. This means that the ultimate effects of specifying a concept one way or another are truly unpredictable at the outset.

Here, we will test the forecasting implications of these alternative conceptualizations of a coup, using the Virtual Strategic Analysis and Forecasting Tool (V-SAFT). V-SAFT, which Lustick Consulting has developed with support from DARPA and ONR, includes a battery of agent-based models representing virtual countries available for interrogation, experimentation, and forecasting. Each model is comprised from (1) a landscape of individual agents heterogeneously characterized to reflect the particular social and political topology of the target country, (2) simple rules of interaction by which agents adopt, discard, and trade politically motivating identities, and (3) broader rules orienting swaths of the landscape toward or away from the political center of power (classified as levels, from ‘dominance’ at the center, to a position of radical opposition at the ‘non-system’ level).

Using our models for Thailand, Pakistan, and Egypt, and keeping with the examples posed above, we operationalized a coup in three ways.

Concept Definitional Criteria Model Operationalization
A Coup
(root concept)
Extra-legal seizure A move from the ‘non-system’ level of the model
Of the chief executive office To a position of political ‘dominance’ in the model
By any state actor By the ‘state’ OR the ‘military’ identity
A Military Coup
(subtype)
Extra-legal seizure A move from the ‘non-system’ level of the model
Of the chief executive office To a position of political ‘dominance’ in the model
By a military actor By the ‘military’ identity exclusively
A Parliamentary Coup
(diminished subtype)
Extra-legal seizure A move from the ‘non-system’ level of the model
Of the chief executive office To a position of political ‘dominance’ in the model
By any unarmed political insider By any ‘political party’ identity, but NOT the state or military identities

Our forecast for each conceptualization of a coup, alongside Ulfelder’s original forecasts, appear in the figure below.

Coup Likelihoods Combined

A first glance tells us that disaggregating the root concept ‘coup’ affects the rank ordering of likelihoods. With the parliamentary coup (diminished subtype) operationalization, the rankings produced by LC’s forecasts match Ulfelder’s. With the simple coup or military coup (subtype) operationalizations, however, Thailand drops in the rankings, behind Pakistan and then Egypt.

Where this kind of concept disaggregation buys us the most explanatory power is where we see forecast divergence. In Ulfelder’s forecast, we see the greatest disagreement between his two models on the forecast for Thailand, indicating a lower level of certainty for forecast accuracy. In LC’s forecasts, we also see the greatest disagreement in Thailand. In other words, how we operationalize and disaggregate the root concept ‘coup’ seems to have the greatest implications in Thailand, compared to Pakistan and Egypt – especially with regard to the diminished subtype. What are the implications in Thailand of such a jump in likelihood for a parliamentary coup, alongside a decrease in the likelihood for a military or traditional coup?

 Attack Likelihood by Coup Type

One of the more puzzling features of the Ukraine case’s coup-ish-ness is its normative implications. If the Ukrainian case is a ‘just coup,’ is that exceptional, or are all parliamentary coups ‘just?’ And would a different type of coup have been unjust? The figure below provides some insight. Here we see the likelihood of subversive anti-state violence—a dependent variable endogenously produced by V-SAFT models—disaggregated by coup type. While the presence of a coup significantly increases the likelihood of anti-state violence, the effect is generally greater for a traditional or military coup. The relationship between coup type and violence is more pronounced in the Thailand case, which is likely due to its larger sample size (as noted in the preceding figure, Thailand generated more parliamentary coups). So, while coups of any kind are rarely peaceful, the diminished subtype that we saw in Ukraine is generally associated with less popular violence. In this way, disaggregating our concept can at times help our analytical thinking inform our moral thinking.

By systematically exploiting theoretically differentiated ABM simulation models, forecasters and analysts can sharpen their self-awareness of concept choice, measure its implications, and ultimately improve the real utility of concept categories for policy makers. To further explore the forecasts produced by V-SAFT’s regular battery of country models (Egypt, Pakistan, Thailand, the Philippines, Bangladesh, Venezuela, Indonesia, Malaysia, and Kenya), please visit this page (registration required) on Lustick Consulting’s web site. For our publications on the application of agent-based modeling approaches to social science, please see this page (no registration required).

The Effervescence of Protest Power

As you probably know if you read this blog, hundreds of thousands of Ukrainians have taken to the streets in frustration over the past week after President Viktor Yanukovich reversed course and decided not to sign an association agreement with the European Union. Meanwhile, thousands of Thais ostensibly aggravated by an amnesty bill that might have opened the door to former prime minister Thaksin Shinawatra’s return have swarmed and occupied several government ministries and are now openly demanding that the army oust the current prime minister.

I’m not going to try to give accounts of these events, which are still growing and morphing, or their causes, which are multifaceted and inevitably ambiguous. Instead, I’d like to underscore what the protests in both countries show us again about how hard it is to convert popular action into political change.

Cheering on the rallies in Kiev earlier today, writer Anne Applebaum tweeted:

She’s right, of course. The problem for the tens of thousands in Ukraine’s and Thailand’s streets is that this is a very hard task. Political power is an amorphous thing, and inertia is its most prevalent form. “Levers of power” is just a metaphor, but that’s the point. The absence of something tangible for protesters to seize and push and wield makes it surprisingly difficult for them to convert their physical mass and emotional energy into the changes in rules and procedures and personnel they usually seek. Protesters can fill squares and topple statues and even swarm the buildings where laws are made, but the social practice of government does not allow them to pick up pen and paper and rewrite the rules while they’re there. When protesters try to do something like that, they are usually ignored or waited out or driven back with violence.

To convert their apparent power into significant political change, protesters usually have to find a way to convince some people on the inside to listen—to rewire the system on their behalf, or to invite their representatives into the control room. In Thailand, anti-government protesters are directing their appeals at military leaders, but so far those leaders aren’t heeding the call. In Ukraine, boxer-cum-presidential hopeful Vladimir Klitschko has called on President Yanukovich to resign, but so far Yanukovich isn’t budging.

Even when they seem to heed the crowd’s call, those insiders have an uncanny knack for bending the arc of politics back toward the status quo ante. In Egypt in 2011, protesters got the military to force Hosni Mubarak from office after decades in power, but the military-led transition that ensued has somehow landed that process not far from where it started.

I don’t mean to be a downer—to suggest that protesters are powerless, or that we can or should measure the value of activism solely by the change it immediately does or does not produce. In a way, you could read this reminder as a backhanded compliment to the activists who do manage to produce significant and durable change. As Frederick Douglass famously suggested, mass activism may not be sufficient to produce institutional change, but it is almost always necessary.

It’s also not clear that the effervescence of protest power is inherently a bad thing. In the case of Thailand, people who value democracy should probably be cheering the failure of a reactionary, elitist movement that’s trying to topple an elected and still reasonably popular government. While Yanukovich’s democratic credentials are shakier, I’m sure there are many Ukrainians who feel the same way about any effort to force their president from office before his term expires. Maybe this tendency toward inertia in politics is the inevitable output of the contraptions we build and task with reconciling our heterogeneous desires for progress and our shared fear of disorder.

Can Venezuela’s Maduro Survive Hyperinflation?

Venezuela is probably sliding into a period of hyperinflation, says the Cato Institute’s Steve Hanke. A picture in a recent blog post of his pretty much tells the story:

venezuela_chart_22

The economic crisis of which this inflationary spiral is just one part has lots of people wondering how long Venezuela’s president, Nicolás Maduro, can hang on to power. Historical evidence on what happens to political leaders during periods of hyperinflation could give us a good starting point for hazarding a prediction on that matter. Best I can tell, though, this isn’t something that’s been studied before, so I decided to scrounge up some some data and take a look.

I started with a table Hanke and Nicolas Krus published in 2012 that identifies all episodes of hyperinflation around the world since the late eighteenth century (here). By their definition, a hyperinflationary episode starts when there is a month in which prices increase by at least 50%, and it ends when the inflation rate drops below that threshold and then stays under it for at least a year. Their table identifies when each episode began and ended and the peak and average daily inflation rates involved. The good news is that this data set exists. The bad news is that it is only posted in PDF form, so I had to type it into a spreadsheet to start working with it.

According to Hanke and Krus’ table, there have been more than 50 spells of hyperinflation around the world in the past couple of centuries. As the plot below shows, virtually all of those occurred the past 100 years in three clusters: one in the 1920s, another in the 1940s, and the last and by far the largest in the 1990s following the disintegration of the USSR and Yugoslavia.

Hyperinflation Episodes around the World, 1790-2012

Hyperinflation Episodes around the World, 1790-2012

The duration of those episodes has varied widely, from a few months or less (many cases) to more than five years (Nicaragua from 1986 until 1991). As you can see in the histogram below, the distribution of durations seems to be bimodal. Most episodes end quickly, but the ones that don’t usually go on to last at least two or three years.

Duration of Hyperinflation Episodes, in Months

Duration of Hyperinflation Episodes, in Months

The average daily rate of inflation in those episodes has varied much less. As the next histogram shows, nearly all of the episodes have involved average daily rates in the low single digits. Cases like Zimbabwe in 2007-2008, when the daily inflation rate averaged nearly 100% (!), are quite rare.

Average Daily Inflation Rates during Episodes of Hyperinflation

Average Daily Inflation Rates during Episodes of Hyperinflation

To analyze the fate of political leaders during these episodes, I used the Archigos data set to create a variable indicating whether or not a country’s chief executive was replaced during or soon after the episode of hyperinflation. Suspecting that those fates would depend, in part, on the nature of a country’s national political regime, I also used a data set I created in a past professional life to add a variable marking whether or not a country’s political regime was democratic when the episode started.

A quick look at a contingency table confirmed my hunches that political leaders often lose their jobs during periods of hyperinflation, but also that the pattern differs across democracies and autocracies. Of the 49 episodes that occurred in cases for which I also had data on leaders’ fates and regime type, leadership changes occurred during or soon after 18 of them (37 percent). Eleven of those changes occurred in the 23 cases that were democracies at the time (48 percent). The other seven leader changes came from the 26 episodes that occurred under authoritarian regimes (27 percent). Based on those data alone, it looks like chief executives in democracies are about as likely to lose their jobs during a hyperinflationary episode as they are to hang on to them, while autocrats face more favorable odds of political survival of roughly 3:1.

Of course, the episodes of hyperinflation aren’t identical. As we saw above, some last a lot longer than others, and some involve much steeper inflation rates. To get a sense of how those things affect the fate of the leaders who preside over these dismal spells, I used the ‘glm‘ command in R to estimate a logistic regression model with my binary leadership-change indicator as the outcome and democracy, episode duration, and average daily inflation rate as the covariates. Guessing that the effects of the latter two covariates might be mediated by regime type, I also included interaction terms representing the products of my democracy indicator and those other two variables.

The model is admittedly crude,* but I think the results are still interesting. According to my estimates, the severity of the episode isn’t systematically associated with variation in the fate of national leaders in either type of political regime. For both democracies and autocracies, the substantive effects of the average daily rate over the course of the hyperinflationary episode were roughly zero.

By contrast, the duration of the episode does seem to matter, but only in autocracies. Democratically elected leaders are relatively vulnerable no matter how long the episode lasts. For their part, autocrats aren’t very likely to get knocked out of office during short episodes, but in episodes that persist for a few years, they are about as likely to get tossed as their democratic counterparts. The plot below shows just how bad it gets for autocrats in long-lasting hyperinflationary episodes, assuming average severity. Part of that’s just the additional exposure—the longer the episode, the more likely we are to see a leader exit office for any reason—but the estimated probabilities we see here are much higher than the base rate of leadership change in authoritarian regimes, so it looks like the extended spell of hyperinflation is probably doing some of the work.

Hyperinflation Episode Duration and the Probability of Leadership Change

Hyperinflation Episode Duration and the Probability of Leadership Change

So what does all this tell us about Maduro’s prospects for political survival, assuming that Venezuela is sliding into a period of hyperinflation? I consider Venezuela’s political regime to be authoritarian, so f I only had these statistics to go by, I would say that Maduro will probably survive the episode, but the chances that he’ll get run out of office will increase the longer the hyperinflation lasts. I’m not an economist, so my best guess at how long Venezuela might suffer under hyperinflation is the average duration from Hanke’s list. That’s a little shy of two years, which would give Maduro odds of about 4:1 to of weathering that storm.

Of course, those statistics aren’t all the information we’ve got. Other things being equal, authoritarian regimes with leaders in their first five years in office—like Venezuela right now—are about three times as likely to transition to democracy as ones with guys who’ve been around for longer, and democratic transitions almost always entail a change at the top. We also know that Maduro so far has been a “boring and muddled” politician, and that there are some doubts about the loyalty he can expect from the military and from other Chavista elites. Putting all of those things together, I’d say that Maduro’s presidency probably won’t last the six years he won in the April 2013 election. Who or what might come next is a whole other question, but as a new leader presiding over an inflationary spiral with weak skills and a shaky coalition, Maduro would seem to have the deck stacked against him.

Data and code for the plots and modeling can be found here and here, respectively.

* To really do this right, I would want to plot survival curves that treat the time from the onset of the hyperinflationary episode to the leader’s exit as the outcome of interest, with right censoring at the episode’s end and regime type as an initial condition. As they say in academese, though, the data preparation that more careful analysis would require was beyond the scope of this blog post. I mean, I’m not Brett Keller.

Challenges in Measuring Violent Conflict, Syria Edition

As part of a larger (but, unfortunately, gated) story on how the terrific new Global Data on Events, Language, and Tone (GDELT) might help social scientists forecast violent conflicts, the New Scientist recently posted some graphics using GDELT to chart the ongoing civil war in Syria. Among those graphics was this time-series plot of violent events per day in Syria since the start of 2011:

Syrian Conflict   New Scientist

Based on that chart, the author of the story (not the producers of GDELT, mind you) wrote:

As Western leaders ponder intervention, the resulting view suggests that the violence has subsided in recent months, from a peak in the third quarter of 2012.

That inference is almost certainly wrong, and why it’s wrong underscores one of the fundamental challenges in using event data—whether it’s collected and coded by software or humans or some combination thereof—to observe the dynamics of violent conflict.

I say that inference is almost certainly wrong because concurrent data on deaths and refugees suggest that violence in Syria has only intensified in past year. One of the most reputable sources on deaths from the war is the Syria Tracker. A screenshot of their chart of monthly counts of documented killings is shown below. Like GDELT, their data also identify a sharp increase in violence in late 2012. Unlike GDELT, their data indicate that the intensity of the violence has remained very high since then, and that’s true even though the process of documenting killings inevitably lags behind the actual violence.

Syria Tracker monthly death counts

We see a similar pattern in data from the U.N. High Commissioner on Refugees (UNHCR) on people fleeing the fighting in Syria. If anything, the flow of refugees has only increased in 2013, suggesting that the violence in Syria is hardly abating.

UNHCR syria refugee plot

The reason GDELT’s count of violent events has diverged from other measures of the intensity of the violence in Syria in recent months is probably something called “media fatigue.” Data sets of political events generally depend on news sources to spot events of interest, and it turns out that news coverage of large-scale political violence follows a predictable arc. As Deborah Gerner and Phil Schrodt describe in a paper from the late 1990s, press coverage of a sustained and intense conflicts is often high when hostilities first break out but then declines steadily thereafter. That decline can happen because editors and readers get bored, burned out, or distracted. It can also happen because the conflict gets so intense that it becomes, in a sense, too dangerous to cover. In the case of Syria, I suspect all of these things are at work.

My point here isn’t to knock GDELT, which is still recording scores or hundreds of events in Syria every day, automatically, using open-source code, and then distributing those data to the public for free. Instead, I’m just trying to remind would-be users of any data set of political events to infer with caution. Event counts are one useful way to track variation over time in political processes we care about, but they’re only one part of the proverbial elephant, and they are inevitably constrained by the limitations of the sources from which they draw. To get a fuller sense of the beast, we need as often as possible to cross-reference those event data with other sources of information. Each of the sources I’ve cited here has its own blind spots and selection biases, but a comparison of trends from all three—and, importantly, an awareness of the likely sources of those biases—is enough to give me confidence that the civil war in Syria is only continuing to intensify. That says something important about Syria, of course, but it also says something important about the risks of drawing conclusions from event counts alone.

PS. For a great discussion of other sources of bias in the study of political violence, see Stathis Kalyvas’ 2004 essay on “The Urban Bias in Research on Civil Wars” (PDF).

I’m Down with Complexity and All, But…

In a recent Scientific American blog post called “Big Data Needs a Big Theory“, Geoffrey West calls for a unified theory of complex systems that will advance our understanding of, and capacity to predict, stasis and change in many domains. Quoting at length:

The digital revolution is driving much of the increasing complexity and pace of life we are now seeing, but this technology also presents an opportunity… With new computational tools and techniques to digest vast, interrelated databases, researchers and practitioners in science, technology, business and government have begun to bring large-scale simulations and models to bear on questions formerly out of reach of quantitative analysis, such as how cooperation emerges in society, what conditions promote innovation, and how conflicts spread and grow.

The trouble is, we don’t have a unified, conceptual framework for addressing questions of complexity. We don’t know what kind of data we need, nor how much, or what critical questions we should be asking. “Big data” without a “big theory” to go with it loses much of its potency and usefulness, potentially generating new unintended consequences.

When the industrial age focused society’s attention on energy in its many manifestations—steam, chemical, mechanical, and so on—the universal laws of thermodynamics came as a response. We now need to ask if our age can produce universal laws of complexity that integrate energy with information. What are the underlying principles that transcend the extraordinary diversity and historical contingency and interconnectivity of financial markets, populations, ecosystems, war and conflict, pandemics and cancer? An overarching predictive, mathematical framework for complex systems would, in principle, incorporate the dynamics and organization of any complex system in a quantitative, computable framework.

We will probably never make detailed predictions of complex systems, but coarse-grained descriptions that lead to quantitative predictions for essential features are within our grasp. We won’t predict when the next financial crash will occur, but we ought to be able to assign a probability of one occurring in the next few years. The field is in the midst of a broad synthesis of scientific disciplines, helping reverse the trend toward fragmentation and specialization, and is groping toward a more unified, holistic framework for tackling society’s big questions.

Not to put too fine a point on it, but I think that agenda is unrealistic.

I agree with West that human social systems are best understood as complex systems in the technical sense of that term (see here). Still, on the possibility of law-like regularities in complex systems that extend to large-scale human social behavior and are usefully predictive, I’m skeptical. It’s hard for me to imagine what those laws would look like, but then I know that my incapacity to understand the universe is not a reliable indicator of the universe’s inherent regularity or intelligibility.protein_network

At the same time, I think West’s analogizing to physics and the laws of thermodynamics ignores the single most-important difference between the “natural” sciences and the social sciences, namely, the (in)ability to perform true experiments. (N.B. Humans and their social interactions are, of course, entirely “natural,” too, but these are the terms we conventionally use.) Social scientists can only observe the systems we study; we can’t repeatedly perturb them in specific ways under tightly controlled conditions and see how things play out.

The impossibility of experimentation means we’re never going to be able to see the counterfactuals we’d need to see to make clear and confident inferences about rules or laws. That doesn’t mean we can’t find some robust patterns, but those patterns will never be anywhere near as universal and specific as the laws of thermodynamics.

The fuzziness of our understanding also means that the patterns we do see will have only modest predictive power at best. Those fuzzy patterns will allow us to assess differences in propensities with some success, as they already do now, but they will not lead us to sharply accurate predictions about the timing and details of change.

More important, those patterns themselves will change over time, as the underlying system continues to evolve. As West suggests, the changes that are creating new opportunities for analysis are themselves products of exponential growth in the complexity of human society. It’s an empirical question, I suppose, but I find it hard to believe that the processes which beget conflicts between states in the middle of the twenty-first century—an age of nukes and mega-cities and deep globalization—will resemble the processes that begat World Wars I and II in all but the most banal ways. And, of course, that’s assuming that states in the conventional sense are even still around.

A Few Suggestions for Social Scientists New to Twitter

Earlier today, one scholar whose work I greatly admire asked another scholar whose work I greatly admire for advice on how to get started on Twitter. I liked Dan’s response, but I thought I’d take Christian’s query as an open invitation to share a few suggestions of my own. So:

Replace the egg with a picture of you. Seriously, don’t even start following people until you’ve done this. It’s not vain; it’s just letting people know that there’s (probably) a real human on the other end, and letting us know something about how you plan to present yourself in this context. Some people can get away with using cartoons or pictures of their pets or kids, but most of us can’t. So, unless you’re trying to make a very specific statement by doing something different, you probably shouldn’t try.

Decide why you’re using Twitter. If your main goal is to use Twitter as a news feed or to follow other peoples’ work, then it’s a really easy tool to use. Just poke around until you find people and organizations that routinely cover the issues that interest you, and follow them. If, however, your goal is to develop a professional audience, then you need to put more thought into what you tweet and retweet, and the rest of my suggestions might be useful.

Pick your niche(s). There are a lot of social scientists on Twitter, and many of them are picky about whom they follow. To make it worth peoples’ while to add you to their feed, pick one or a few of your research interests and focus almost all of your tweets and retweets on them. For example, I’ve tried to limit my tweets to the topics I blog about: democratization, coups, state collapse,  forecasting, and a bit of international relations. When I was new to Twitter, I focused especially on democratization and forecasting because those weren’t topics other people were tweeting much about at the time. I think that differentiation made it easier for people to attach an identity to my avatar, and to understand what they would get by following me that they weren’t already getting from the 500 other accounts in their feeds.

Keep the tweet volume low, at least at the start. For a long time, I tried to limit myself to two or three tweets per Twitter session, usually once or twice per day. That made me think carefully about what I tweeted, (hopefully) keeping the quality higher and preventing me from swamping peoples’ feeds, a big turnoff for many.

Don’t just share the news; augment it. If you’re tweeting a news story or journal article or something, use a short quote or comment that crystallizes the story or tells us something about why you think it’s worth reading. In other words, try to add value. I usually lead with the title, then insert the link, then hang the quote or comment at the end, like this:

But, of course, there are lots of ways to do this. You can also drop the title entirely, like this recent one from Joshua Kucera that got me laughing:

Keep it professional.  If you’re thinking of Twitter as an extension of your work, don’t tweet about personal stuff. This is especially important when you’re new to the medium. The occasional reference to your life outside the office can help people feel more connected to you, but please err on the side of reticence. I have chosen not to follow or unfollowed many people because the interesting stuff in their feed was overwhelmed by the personal and trivial (and sometimes just downright gross). At some point, all that jetsam gets in the way of the information I’m actually looking for, so I choose to cut it off.

Related to the previous suggestion, be polite. In theory, this should go without saying, but, hey, this is the Internet. If you’re using Twitter for professional purposes, I think it makes sense to use the same language and demeanor you’d use in the office or at a professional conference. That can include humor and the occasional personal tidbit you’d share in a hallway conversation, but probably not the bar talk, and definitely not the post-conference conversations with your confidantes. It most definitely does not include nastiness or pettiness.

Be generous. Don’t retweet something under your own handle just to troll for RTs. If you want to share something someone else already shared, just pass along his or her tweet. The exception to this rule is when you’re going to add your own comment. Then just be sure to acknowledge the source with a via or h/t (hat tip). If a bunch of people already shared something so you’re not sure whom to credit, the answer is, Don’t share it again.

If you modify someone’s tweet at all before passing it along, use MT. This is a Twitter pet peeve of mine. RT (retweet) should only be used when what follows is a verbatim replication of the original. If you change anything—abbreviate, drop a comma, whatever—use MT (modified tweet) instead.

Finally, know that it’s addictive. I don’t mean fun-and-time-consuming addictive; I mean addictive addictive, like nicotine and booze. Before you dive in, it’s worth considering how that addiction might negatively affect your life and how you plan to deal with it. Just because lots of people do it doesn’t mean it’s good for you. The time you spend on Twitter is time you could have spent doing something else. If that something else is more important and you’re prone to addiction, be careful.

Development as Ideology

In a blog post yesterday, Duke economist Marc Bellemare responded to a recent contrarian piece on Foreign Policy‘s Democracy Lab about development trends in Africa. In the FP essay, Rick Rowden argued that recent claims of “Africa rising” ring hollow because the expansions of GDP and trade on which those claims are based aren’t driven by industrialization. Bellemare’s having none of it:

Not everyone agrees as to what “development” means, but for most economists, development means increased standards of living, which are best measured via economic statistics such as gross domestic product (GDP) per capita, which may or may not reflect growth in the manufacturing and services sector of the economy.

I think Marc’s right that it’s more useful to define (human social) development in ways that are agnostic of specific causes, and that many professional economists nowadays think in those terms. At the same time, I get a little uneasy whenever development is linked tightly to GDP growth. Being acquainted with Marc, I suspect that his aim in doing so is simply to make empirical analysis of development more tractable. Still, it’s also true that there’s a powerful strain of technocratic thinking in some quarters of economics, one that prioritizes growth over all other things, and this myopia can sometimes become pernicious. In a recent piece for The Atlantic, for example, Armin Rosen accuses Jeffrey Sachs, a leading candidate for president of the World Bank not so long ago, of turning a blind eye toward the human-rights violations of authoritarian leaders in his pursuit of improved standards of living in Africa and elsewhere. The technocratic mindset was also on full display in a now-failed plan for charter cities in Honduras pushed by economist Paul Romer, and it’s a recurring theme in the columns of Thomas Friedman, whom development professionals love to hate.

In fact, even the driest definitions of human social development will inevitably carry a strong whiff of ideology, because the standards we set and the ways we measure progress toward them shape our behavior. Any definition of development implicitly or explicitly prioritizes some vision of the good life over others, and those visions generally entail some specific ideas about how to get there. The choice to include or expunge industrialization from a definition of “development,” for example, can influence what kinds of policies governments adopt in the pursuit or distribution of aid and loans tied to those metrics, and those policies can have vast consequences.

This is not a concern that’s unique to economics. In American political science, at least, when someone talks about “development,” they often mean to invoke a cluster of economic, social, and political changes that Seymour Martin Lipset called “modernization.” That cluster includes the improvements in standards of living that Bellemare emphasizes, but it also includes the industrialization Rowden spotlights, along with other things like urbanization, education, the spread of liberal values, and, perhaps most important, the emergence of a specific form of political democracy. In short, to “develop” was to follow the specific trail of socioeconomic transformation from primitivism into modernity that was blazed by Europe in the 19th and 20th centuries, and that emphasis on mimicry has profoundly affected the ways the U.S. and Europe have tried to promote development.

When Lipset was writing, modernization theory’s chief intellectual and political competitor was Marxism. For Marxist theorists, Europe wasn’t a model to emulate; it was a bastion of economic inequality and plutocratic “democracy” that would eventually and inevitably collapse under the weight of its internal contradictions. Industrialization was a critical feature of capitalism’s late stages that enabled maximal exploitation of labor by capital. It was both bad and good—bad because of the depths of exploitation it entailed, but good because it meant the end was nigh.

Of course, it was the USSR and its European client regimes that actually collapsed, and in the two decades since, almost all of the purportedly communist states left in the world have abandoned their commitment to Marxism and adopted variations on the capitalist theme instead. That turn in international political economy has hardly killed class-conscious theory, however. Although the strict Hegelian version of Marxism is rarely seen nowadays, the concern with economic inequality and its political consequences remains a central theme in leftist politics. Flip this concern around, and we arrive at yet another definition of development. For many leftists today, development is about the spread of social justice, and the essence of social justice is not wealth but fairness. Industrialization, electoral democracy, and economic expansion are not things to be valued in themselves but means (or, in some cases, obstacles) to these deeper ends.

Economic growth, modernization, and social justice are probably the three most prominent conceptualizations of development today, but they certainly aren’t the only ones. You might not think of libertarianism as a theory of development, but in an important sense it is. For libertarians, the good life is understood as one in which individuals are free to do as they please within only the sparest of constraints. Here again, industry, democracy, and growth are all beside the point. Liberty is the goal, and social and political changes that expand freedom can be understood as developmental gains. This idea finds one of its sharpest expressions in the Heritage Foundation’s Index of Economic Freedom, which that organization deliberately presents as an alternative to conventional measures focused on specific outcomes like poverty and education rates.

All of which is a very long-winded reminder that no conceptualization of development can exist without an ideological foundation. That would be like a shadow without a light source, a hole when there is no surface. To talk about “development” is to set goals for human social improvement, and the choices we make in setting those goals are inherently normative. We can’t escape this circle, so we might as well be explicit about it.

PS. Yes, I’m aware this has all been said a thousand times before, and often better. I decided to write the post anyway because it helped me organize some of my own thinking on the subject, and because the occasional reminder still can’t hurt.

Follow

Get every new post delivered to your Inbox.

Join 7,071 other followers

%d bloggers like this: