In Praise of a Measured Response to the Ukraine Crisis

Yesterday afternoon, I tweeted that the Obama administration wasn’t getting enough credit for its measured response to the Ukraine crisis so far, asserting that sanctions were really hurting Russia and noting that “we”—by which I meant the United States—were not directly at war.

Not long after I said that, someone I follow tweeted that he hadn’t seen a compelling explanation of how sanctions are supposed to work in this case. That’s an important question, and one I also haven’t seen or heard answered in depth. I don’t know how U.S. or European officials see this process beyond what they say in public, but I thought I would try to spell out the logic as a way to back up my own assertion in support of the approach the U.S. and its allies have pursued so far.

I’ll start by clarifying what I’m talking about. When I say “Ukraine crisis,” I am referring to the tensions created by Russia’s annexation of Crimea and its evident and ongoing support for a separatist rebellion in eastern Ukraine. These actions are only the latest in a long series of interactions with the U.S. and Europe in Russia’s “near abroad,” but their extremity and the aggressive rhetoric and action that has accompanied them have sharply amplified tensions between the larger powers that abut Ukraine on either side. For the first time in a while, there has been open talk of a shooting war between Russia and NATO. Whatever you make of the events that led to it and however you assign credit or blame for them, this state of affairs represents a significant and undesirable escalation.

Faced with this crisis, the U.S. and its NATO allies have three basic options: compel, cajole, or impel.

Compel in this case means to push Russia out of Ukraine by force—in other words, to go to war. So far, the U.S. and Europe appear to have concluded—correctly, in my opinion—that Russia’s annexation of Crimea and its support for separatists in eastern Ukraine does not warrant a direct military response. The likely and possible costs of war between two nuclear powers are simply too great to bear for the sake of Ukraine’s autonomy or territorial integrity.

Cajoling would mean persuading Russian leaders to reverse course through positive incentives—carrots of some kind. It’s hard to imagine what the U.S. and E.U. could offer that would have the desired effect, however. Russian leaders consider Ukraine a vital interest, and the West has nothing comparably valuable to offer in exchange. More important, the act of making such an offer would reward Russia for its aggression, setting a precedent that could encourage Russia to grab for more and could also affect other country’s perceptions of the U.S.’s tolerance for seizures of territory.

That leaves impel—to impose costs on Russia to the point where its leaders feel obliged to change course. The chief tool that U.S. and European leaders have to impose costs on Russia are economic and financial sanctions. Those leaders are using this tool, and it seems to be having the desired effect. Sanctions are encouraging capital flight, raising the costs of borrowing, increasing inflation, and slowing Russia’s already-anemic economic growth (see here and here for some details). Investors, bankers, and consumers are partly responding to the specific constraints of sanctions, but they are also responding to the broader economic uncertainty associated with those sanctions and the threat of wider war they imply. “It’s pure geopolitical risk,” one analyst told Bloomberg.

These costs can directly and indirectly shape Russian policy. They can directly affect Russian policy if and as the present leadership comes to view them as unbearable, or at least not worth the trade-offs against other policy objectives. That seems unlikely in the short term but increasingly likely over the long term, if the sanctions are sustained and markets continue to react so negatively. Sustained capital flight, rising inflation, and slower growth will gradually shrink Russia’s domestic policy options and its international power by eroding its fiscal health, and at some point these costs should come to outweigh the putative gains of territorial expansion and stronger leverage over Ukrainian policy.

These costs can also indirectly affect Russian policy by increasing the risk of internal instability. In authoritarian regimes, significant reforms usually occur in the face of popular unrest that may or may not be egged on by elites who defect from the ruling coalition. We are already seeing signs of infighting among regime insiders, and rising inflation and slowing growth should increase the probability of popular unrest.

To date, sanctions have not dented Putin’s soaring approval rating, but social unrest is not a referendum. Unrest only requires a small but motivated segment of the population to get started, and once it starts, its very occurrence can help persuade others to follow. I still wouldn’t bet on Putin’s downfall in the near future, but I believe the threat of significant domestic instability is rising, and I think that Putin & co. will eventually care more about this domestic risk than the rewards of continued adventurism abroad. In fact, I think we see some evidence that Putin & co. are already worrying more about this risk in their ever-expanding crackdown on domestic media and their recent moves to strengthen punishment for unauthorized street rallies and, ironically, calls for separatism. Even if this mobilization does not come, the increased threat of it should weigh on the Russian administration’s decision-making.

In my tweet on the topic, I credited the Obama administration for using measured rhetoric and shrewd policy in response to this crisis. Importantly, though, the success of this approach also depends heavily on cooperation among the U.S. and the E.U., and that seems to be happening. It’s not clear who deserves the credit for driving this process, but as one anonymous tweeter pointed out, the downing of flight MH17 appears to have played a role in deepening it.

Concerns are growing that sanctions may, in a sense, be too successful. Some observers fear that apparent capitulation to the U.S. and Europe would cost Russian leaders too much at home at a time when nationalist fervor has reached fever pitch. Confronted with a choice between wider war abroad or a veritable lynch mob at home, Putin & co. will, they argue, choose the former.

I think that this line of reasoning overstates the extent to which the Russian administration’s hands are tied at home. Putin & co. are arguably no more captive to the reinvigorated radical-nationalist fringe than they were to the liberal fringe that briefly threatened to oust them after the last presidential election.

Still, it is at least a plausible scenario, and the U.S. and E.U. have to be prepared for the possibility that Russian aggression will get worse before it gets better. This is where rhetorical and logistical efforts to bolster NATO are so important, and that’s just what NATO has been doing. NATO is predicated on a promise of collective defense; an attack on any one member state is regarded as an attack on all. By strengthening Russian policy-makers’ beliefs that this promise is credible, NATO can lead them to fear that escalations beyond certain thresholds will carry extreme costs and even threaten their very survival. So far, that’s just what the alliance has been doing with a steady flow of words and actions. Russian policy-makers could still choose wider war for various reasons, but theory and experience suggest that they are less likely to do so than they would be in the absence of this response.

In sum, given a short menu of unpalatable options, I think that the Obama administration and its European allies have chosen the best line of action and, so far, made the most of it. To expect Russia quickly to reverse course by withdrawing from Crimea and stopping its rabble-rousing in eastern Ukraine without being compelled by force to do so is unrealistic. The steady, measured approach the U.S. and E.U. have adopted appears to be having the intended effects. Russia could still react to the rising structural pressures on it by lashing out, but NATO is taking careful steps to discourage that response and to prepare for it if it comes. Under such lousy circumstances, I think this is about as well as we could expect the Obama administration and its E.U. counterparts to do.

Uncertainty About How Best to Convey Uncertainty

NPR News ran a series of stories this week under the header Risk and Reason, on “how well we understand and act on probabilities.” I thought the series nicely represented how uncertain we are about how best to convey forecasts to people who might want to use them. There really is no clear standard here, even though it is clear that the choices we make in presenting forecasts and other statistics on risks to their intended consumers strongly shape what they hear.

This uncertainty about how best to convey forecasts was on full display in the piece on how CIA analysts convey predictive assessments (here). Ken Pollack, a former analyst who now teaches intelligence analysis, tells NPR that, at CIA, “There was a real injunction that no one should ever use numbers to explain probability.” Asked why, he says that,

Assigning numerical probability suggests a much greater degree of certainty than you ever want to convey to a policymaker. What we are doing is inherently difficult. Some might even say it’s impossible. We’re trying to protect the future. And, you know, saying to someone that there’s a 67 percent chance that this is going to happen, that sounds really precise. And that makes it seem like we really know what’s going to happen. And the truth is that we really don’t.

In that same segment, though, Dartmouth professor Jeff Friedman, who studies decision-making about national security issues, says we should provide a numeric point estimate of an event’s likelihood, along with some information about our confidence in that estimate and how malleable it may be. (See this paper by Friedman and Richard Zeckhauser for a fuller treatment of this argument.) The U.S. Food and Drug Administration apparently agrees; according to the same NPR story, the FDA “prefers numbers and urges drug companies to give numerical values for risk—and to avoid using vague terms such as ‘rare, infrequent and frequent.'”

Instead of numbers, Pollack advocates for using words: “Almost certainly or highly likely or likely or very unlikely,” he tells NPR. As noted by one of the other stories in the series (here), however—on the use of probabilities in medical decision-making—words and phrases are ambiguous, too, and that ambiguity can be just as problematic.

Doctors, including Leigh Simmons, typically prefer words. Simmons is an internist and part of a group practice that provides primary care at Mass General. “As doctors we tend to often use words like, ‘very small risk,’ ‘very unlikely,’ ‘very rare,’ ‘very likely,’ ‘high risk,’ ” she says.

But those words can be unclear to a patient.

“People may hear ‘small risk,’ and what they hear is very different from what I’ve got in my mind,” she says. “Or what’s a very small risk to me, it’s a very big deal to you if it’s happened to a family member.

Intelligence analysts have sometimes tried to remove that ambiguity by standardizing the language they use to convey likelihoods, most famously in Sherman Kent’s “Words of Estimative Probability.” It’s not clear to me, though, how effective this approach is. For one thing, consumers are often lazy about trying to understand just what information they’re being given, and templates like Kent’s don’t automatically solve that problem. This laziness came across most clearly in NPR’s Risk and Reason segment on meteorology (here). Many of us routinely consume probabilistic forecasts of rainfall and make decisions in response to them, but it turns out that few of us understand what those forecasts actually mean. With Kent’s words of estimative probability, I suspect that many readers of the products that use them haven’t memorized the table that spells out their meaning and don’t bother to consult it when they come across those phrases, even when it’s reproduced in the same document.

Equally important, a template that works well for some situations won’t necessarily work for all. I’m thinking in particular of forecasts on the kinds of low-probability, high-impact events that I usually analyze and that are essential to the CIA’s work, too. Here, what look like small differences in probability can sometimes be very meaningful. For example, imagine that it’s August 2001 and you’ve three different assessments of the risk of a major terrorist attack on U.S. soil in the next few months. One pegs the risk at 1 in 1,000; another at 1 in 100; and another at 1 in 10. Using Kent’s table, all three of those assessments would get translated into a statement that the event is “almost certainly not” going to happen, but I imagine that most U.S. decision-makers would have felt very differently about risks of 0.1%, 1%, and 10% with a threat of that kind.

There are lots of rare but important events that inhabit this corner of the probability space: nuclear accidents, extreme weather events, medical treatments, and mass atrocities, to name a few. We could create a separate lexicon for assessments in these areas, as the European Medicines Agency has done for adverse reactions to medical therapies (here, via NPR). I worry, though, that we ask too much of consumers of these and other forecasts if we expect them to remember multiple lexicons and to correctly code-switch between them. We also know that the relevant scale will differ across audiences, even on the same topic. For example, an individual patient considering a medical treatment might not care much about the difference between a mortality risk of 1 in 1,000 and 1 in 10,000, but a drug company and the regulators charged with overseeing them hopefully do.

If there’s a general lesson here, it’s that producers of probabilistic forecasts should think carefully about how best to convey their estimates to specific audiences. In practice, that means thinking about the nature of the decision processes those forecasts are meant to inform and, if possible, trying different approaches and checking to find out how each is being understood. Ideally, consumers of those forecasts should also take time to try to educate themselves on what they’re getting. I’m not optimistic that many will do that, but we should at least make it as easy as possible for them to do so.

Indonesia’s Elections Offer Some Light in the Recent Gloom

The past couple of weeks have delivered plenty of terrible news, so I thought I would take a moment to call out a significant positive development: Indonesia held a presidential election early this month; there were no coup attempts and little violence associated with that balloting; and the contest was finally won by the guy who wasn’t threatening to dismantle democracy.

By my reckoning, this outcome should increase our confidence that Indonesia now deserves to be called a consolidated democracy, where “consolidated” just means that the risk of a reversion to authoritarian rule is low. Democracies are most susceptible to those reversions in their first 15–20 years (here and here), especially when they are poor and haven’t yet seen power passed from one party to another (here).

Indonesia now looks reasonably solid on all of those counts. The current democratic episode began nearly 15 years ago, in 1999, and the country has elected three presidents from as many parties since then—four if we count the president-elect. Indonesia certainly isn’t a rich country, but it’s not exactly poor any more, either. With a GDP per capita of approximately $3,500, it now lands near the high end of the World Bank’s “lower middle income” tier. Together, those features don’t describe a regime that we would expect to be immune from authoritarian reversal, but the elections that just occurred put that system through a major stress test, and it appears to have passed.

Some observers would argue that the country’s democratic regime already crossed the “consolidated” threshold years ago. When I described Indonesia as a newly consolidated democracy on Twitter, Indonesia specialist Jeremy Menchik noted that colleagues William Liddle and Saiful Mujani had identified Indonesia as being consolidated since 2004 and said that he agreed with them. Meanwhile, democratization experts often use the occurrence of one or two peaceful transfers of power as a rule of thumb for declaring democracies consolidated, and Indonesia had passed both of those tests before the latest election campaign even began.

Of course, it’s easy to say in hindsight that the risk of an authoritarian reversal in Indonesia around this election was low. We shouldn’t forget, though, that there was a lot of anxiety during the campaign about how the eventual loser, Prabowo Subianto, might dismantle democracy if he were elected, and in the end he only lost by a few percentage points. What’s more, the kind of “reforms” at which Prabowo hinted are just the sorts of things that have undone many other attempts at democracy in the past couple of decades. There were also rumors of coup plots, especially during the nerve-wracking last few weeks of the campaign until the official results were announced (see here, for example). Some seasoned observers of Indonesian politics with whom I spoke were confident at the time that those plots would not come to pass, but the fact that those rumors existed and were anxiously discussed in some quarters suggests that they were at least plausible, even if they weren’t probable. Last but not least, statistical modeling by Milan Svolik suggests that a middle-income presidential democracy like Indonesia’s won’t really be “cured” of its risk of authoritarian reversal until it gets much wealthier (see the actuarial tables on p. 43 in this excellent paper, which was later published in the American Political Science Review).

Even bearing those facts and Milan’s tables in mind, I think it’s fair to say that Indonesia now qualifies as a consolidated democracy, in the specific sense that the risk of an authoritarian reversal is now quite small and will remain so. If that’s right, then four of the world’s five most populous countries now fit under that label. The democratic regimes in India, the United States, Indonesia, and Brazil—roughly 2 billion citizens among them—all have lots of flaws, but the increased prevalence and persistence of democracy among the world’s largest countries is still a very big deal in the long course of human affairs. And, who knows, maybe China will finally join them in the not-too-distant future?

In Applied Forecasting, Keep It Simple

One of the lessons I think I’ve learned from the nearly 15 years I’ve spent developing statistical models to forecast rare political events is: keep it simple unless and until you’re compelled to do otherwise.

The fact that the events we want to forecast emerge from extremely complex systems doesn’t mean that the models we build to forecast them need to be extremely complex as well. In a sense, the unintelligible complexity of the causal processes relieves us from the imperative to follow that path. We know our models can’t even begin to capture the true data-generating process. So, we can and usually should think instead about looking for measures that capture relevant concepts in a coarse way and then use simple model forms to combine those measures.

A few experiences and readings have especially shaped my thinking on this issue.

  • When I worked on the Political Instability Task Force (PITF), my colleagues and I found that a logistic regression model with just four variables did a pretty good job assessing relative risks of a few forms of major political crisis in countries worldwide (see here, or ungated here). In fact, one of the four variables in that model—an indicator that four or more bordering countries have ongoing major armed conflicts—has almost no variance, so it’s effectively a three-variable model. We tried adding a lot of other things that were suggested by a lot of smart people, but none of them really improved the model’s predictive power. (There were also a lot of things we couldn’t even try because the requisite data don’t exist, but that’s a different story.)
  • Toward the end of my time with PITF, we ran a “tournament of methods” to compare the predictive power of several statistical techniques that varied in their complexity, from logistic regression to Bayesian hierarchical models with spatial measures (see here for the write-up). We found that the more complex approaches usually didn’t outperform the simpler ones, and when they did, it wasn’t by much. What mattered most for predictive accuracy was finding the inputs with the most forecasting power. Once we had those, the functional form and complexity of the model didn’t make much difference.
  • As Andreas Graefe describes (here), models that assign equal weights to all predictors often forecast at least as accurately as multiple regression models that estimate weights from historical data. “Such findings have led researchers to conclude that the weighting of variables is secondary for the accuracy of forecasts,” Graefe writes. “Once the relevant variables are included and their directional impact on the criterion is specified, the magnitudes of effects are not very important.”

Of course, there will be some situations in which complexity adds value, so it’s worth exploring those ideas when we have a theoretical rationale and the coding skills, data, and time needed to pursue them. In general, though, I am convinced that we should always try simpler forms first and only abandon them if and when we discover that more complex forms significantly increase forecasting power.

Importantly, the evidence for that judgment should come from out-of-sample validation—ideally, from forecasts made about events that hadn’t yet happened. Models with more variables and more complex forms will often score better than simpler ones when applied to the data from which they were derived, but this will usually turn out to be a result of overfitting. If the more complex approach isn’t significantly better at real-time forecasting, it should probably be set aside until it does.

Oh, and a corollary: if you have to choose between a) building more complex models, or even just applying lots of techniques to the same data, and b) testing other theoretically relevant variables for predictive power, do (b).

Russia Throws Cuba a Lifeline

Russia has just reinvigorated its relationship with Cuba, and I suspect that this renewed friendship of convenience will help Cuba’s Communist regime stick around longer than it would have without it.

A few things happened, all apparently part of an elaborate quid pro quo. First, while visiting Cuba last week, Russian president Vladimir Putin announced that his country was forgiving nearly all of Cuba’s lingering Soviet-era debt to Russia, or more than $30 billion. Then, a few days later, reports emerged that Cuba had agreed to allow Russia to re-open a large Soviet-era intelligence-gathering facility used to surveil the United States during the Cold War. While in Havana, Putin also spoke of reviving broader military and technological cooperation with Cuba, although he did not say exactly what that would entail. Last but not least, Russia and Cuba reportedly also signed some significant economic contracts, including ones that would allow Russian oil companies to explore Cuban waters.

Putin’s government seems to be responding in kind to what it perceives as a deepening  U.S. threat on its own borders, and this is important in its own right. As a specialist on the survival and transformation of authoritarian regimes, though, I am also interested in how this reinvigorated relationship affects prospects for political change in Cuba.

Consolidated single-party regimes, like Cuba’s, are the most durable kind of autocracies, but when they do break down, it’s usually an economic or fiscal crisis that sets the process in motion. Slumping state revenues shrink the dole that encourages various factions within the party to stay loyal to the ruling elite, while wider economic problems also give ordinary citizens stronger motivations to demand reform. When frustrated citizens and disgruntled insiders find each other, the effect can be especially potent. Economic crisis doesn’t guarantee the collapse of single-party regimes, but it does significantly increase the probability of its occurrence.

The Soviet Union bankrolled Havana for many years, and the Cuban economy has been limping along since that funding stream disappeared along with the country that provided it. In 2o11, the Communist Party of Cuba finally responded to that malaise as formal theory leads us to expect that it would: by experimenting with some limited forms of economic liberalization. These reforms are significant, but as far as I can tell, they have not yet led to the kind of economic renewal that would give the ruling party a serious boost.

One of the reasons the Cuban regime managed to delay those reforms for long was the largesse it received from its close friends in Venezuela. As I discussed in a post here last year, Hugo Chavez’s government used its oil boom to help finance the Cuban regime at a time when Havana would otherwise have been hard pressed to search for new sources of revenue.

With Hugo Chavez dead and Venezuela’s economy in crisis, however, this support has become unreliable. I had expected this uncertainty to increase pressure on the Communist Party of Cuba to expand its liberalization in search of new revenues, and for that expanded liberalization, in turn, to improve prospects for popular mobilization and elite defections that could lead to broader political reforms.

The renewed embrace from Russia now has me revisiting that expectation. The forgiveness of more than $30 billion in debt should provide an immediate boost to Cuba’s finances, but I’m also intrigued by the talk of new oil concessions. For years, the Cuban government has seemed to be hoping that hydrocarbons under its waters would provide it with a new fiscal lifeline. That hasn’t happened yet, but it sounds like Russia and Havana increasingly see prospects for mutual gains in this sphere. Of course, it will also be important to see what other forms of economic and military support are on offer from Moscow and how quickly they might arrive.

None of these developments magically resolves the fundamental flaws in Cuba’s political economy, and so far the government shows no signs of rolling back the process of limited liberalization it has already begun. What’s more, Russia also has economic problems of its own, so it’s not clear how much help it can offer and how long it will be able to sustain that support. Even so, these developments probably do shrink the probability that the Cuban economy will tip soon into a deeper crisis, and with it the near-term prospects for a broader political transformation.

We Are All Victorians

“We have no idea, now, of who or what the inhabitants of our future might be. In that sense, we have no future. Not in the sense that our grandparents had a future, or thought they did. Fully imagined cultural futures were the luxury of another day, one in which ‘now’ was of some greater duration. For us, of course, things can change so abruptly, so violently, so profoundly, that futures like our grandparents’ have insufficient ‘now’ to stand on. We have no future because our present is too volatile… We have only risk management. The spinning of the given moment’s scenarios. Pattern recognition.”

That’s the fictional Hubertus Bigend sounding off in Chapter Six of William Gibson’s fantastic 2003 novel. Gibson is best known as an author of science fiction set in the not-too-distant future. As that passage suggests, though, he is not uniquely interested in looking forward. In Gibson’s renderings, future and past might exist in some natural sense, but our ideas of them can only exist in the present, which is inherently and perpetually liminal.

In Chapter Six, the conversation continues:

“Do we have a past, then?” Stonestreet asks.

“History is a best-guess narrative about what happened and when,” Bigend says, his eyes narrowing. “Who did what to whom. With what. Who won. Who lost. Who mutated. Who became extinct.”

“The future is there,” Cayce hears herself say, “looking back at us. Trying to make sense of the fiction we will have become. And from where they are, the past behind us will look nothing at all like the past we imagine behind us now.”

“You sound oracular.” White teeth.

“I only know that the one constant in history is change: The past changes. Our version of the past will interest the future to about the extent we’re interested in in whatever the past the Victorians believed in. It simply won’t seem very relevant.”

I read that passage and I picture a timeline flipped vertical and frayed at both ends. Instead of a flow of time from left to right, we have only the floating point of the present, with ideas about the future and past radiating outwards and nothing to which we can moor any of it.

In a recent interview with David Wallace-Wells for Paris Review, Gibson revisits this theme when asked about science fiction as futurism.

Of course, all fiction is speculative, and all history, too—endlessly subject to revision. Particularly given all of the emerging technology today, in a hundred years the long span of human history will look fabulously different from the version we have now. If things go on the way they’re going, and technology keeps emerging, we’ll eventually have a near-total sorting of humanity’s attic.

In my lifetime I’ve been able to watch completely different narratives of history emerge. The history now of what World War II was about and how it actually took place is radically different from the history I was taught in elementary school. If you read the Victorians writing about themselves, they’re describing something that never existed. The Victorians didn’t think of themselves as sexually repressed, and they didn’t think of themselves as racist. They didn’t think of themselves as colonialists. They thought of themselves as the crown of creation.

Of course, we might be Victorians, too.

Of course we are. How could we not be?

That idea generally fascinates me, but it also specifically interests me as a social scientist. As discussed in a recent post, causal inference in the social sciences depends on counterfactual reasoning—that is, imagining versions of the past and future that we did not see.

Gibson’s rendering of time reminds us that this is even harder than we like to pretend. It’s not just that we can’t see the alternative histories we would need to compare to our lived history in order to establish causality with any confidence. We can’t even see that lived history clearly. The history we think we see is a pattern that is inexorably constructed from materials available in the present. Our constant disdain for most past versions of those renderings should give us additional pause when attempting to draw inferences from current ones.

The Ethics of Political Science in Practice

As citizens and as engaged intellectuals, we all have the right—indeed, an obligation—to make moral judgments and act based on those convictions. As political scientists, however, we have a unique set of potential contributions and constraints. Political scientists do not typically have anything of distinctive value to add to a chorus of moral condemnation or declarations of normative solidarity. What we do have, hopefully, is the methodological training, empirical knowledge and comparative insight to offer informed assessments about alternative courses of action on contentious issues. Our primary ethical commitment as political scientists, therefore must be to get the theory and the empirical evidence right, and to clearly communicate those findings to relevant audiences—however unpalatable or inconclusive they might be.

That’s a manifesto of sorts, nested in a great post by Marc Lynch at the Monkey Cage. Marc’s post focuses on analysis of the Middle East, but everything he writes generalizes to the whole discipline.

I’ve written a couple of posts on this theme, too:

  • This Is Not a Drill,” on the challenges of doing what Marc proposes in the midst of fast-moving and politically charged events with weighty consequences; and
  • Advocascience,” on the ways that researchers’ political and moral commitments shape our analyses, sometimes but not always intentionally.

Putting all of those pieces together, I’d say that I wholeheartedly agree with Marc in principle, but I also believe this is extremely difficult to do in practice. We can—and, I think, should—aspire to this posture, but we can never quite achieve it.

That applies to forecasting, too, by the way. Coincidentally, I saw this great bit this morning in the Letter from the Editors for a new special issue of The Appendix, on “futures of the past”:

Prediction is a political act. Imagined futures can be powerful tools for social change, but they can also reproduce the injustices of the present.

Concern about this possibility played a role in my decision to leave my old job, helping to produce forecasts of political instability around the world for private consumption by the U.S. government. It is also part of what attracts me to my current work on a public early-warning system for mass atrocities. By making the same forecasts available to all comers, I hope that we can mitigate that downside risk in an area where the immorality of the acts being considered is unambiguous.

As a social scientist, though, I also understand that we’ll never know for sure what good or ill effects our individual and collective efforts had. We won’t know because we can’t observe the “control” worlds we would need to confidently establish cause and effect, and we won’t know because the world we seek to understand keeps changing, sometimes even in response to our own actions. This is the paradox at the core of applied, empirical social science, and it is inescapable.

Another Chicken Little Post on China

Last fall, I described what I saw as an “accumulating risk of crisis” in China. Recent developments in two parts of the country only reinforce my sense that the Communist Party of China (CPC) is entering a period during which it will find it increasingly hard to sustain its monopoly on state authority.

The first part of the country drawing fresh attention is Hong Kong, where pro-democracy activists have mobilized a new nonviolent challenge to the Party’s authority in spite of the center’s pointed efforts to discourage them. Organizing under the Occupy Central label, these activists recently held an unofficial referendum that drew nearly 800,000 voters who overwhelmingly endorsed proposals that would allow the public to nominate candidates for elections in 2017—an idea that Beijing has repeatedly and unequivocally rejected. Today, on 1 July, tens of thousands of people marched into the city’s center to press those same demands.

1 July 2014 rally in Hong Kong (AP via BBC News)

The 1 July rally looks set to be one of the island’s largest protests in years, and it comes only weeks after Beijing issued a white paper affirming its “comprehensive jurisdiction” over Hong Kong. Although the official line since the 1997 handover has been “one country, two systems,” the expectation has generally been that national leaders would only tolerate differences that didn’t directly challenge their authority, and the new white paper made that implicit policy a bit clearer. Apparently, though, many Hong Kong residents aren’t willing to leave that assertion unchallenged, and the resulting conflict is almost certain to persist into and beyond those 2017 elections, assuming Beijing doesn’t concede the point before then.

The second restive area is Xinjiang Uyghur Autonomous Region, where Uyghurs have agitated for greater autonomy or outright independence since the area’s incorporation into China in 1949. Over the past year or so, the pace of this conflict has intensified again.

The Chinese government describes this conflict as a fight against terrorism, and some of the recent attacks—see here and here, for example—have targeted and killed large numbers of civilians. As Assaf Moghadam argues in a recent blog post, however, the line between terrorism and insurgency is almost always blurry in practice. Terrorism and insurgency—and, for that matter, campaigns of nonviolent resistance—are all tactical variations on the theme of rebellion. In Xinjiang, we see evidence of a wider insurgency in recent attacks on police stations and security checkpoints, symbols of the “occupying power” and certainly not civilian targets. Some Uyghurs have also engaged in nonviolent protests, although when they have, the police have responded harshly.

In any case, the tactical variation and increased pace and intensity of the clashes leads me to believe that this conflict should now be described as a separatist rebellion, and that this rebellion now poses a significant challenge to the Communist Party. Uyghurs certainly aren’t going to storm the capital, and they are highly unlikely to win sovereignty or independence for Xinjiang as long as the CPC still rules. Nevertheless, the expanding rebellion is taxing the center, and it threatens to make Party leaders look less competent than they would like.

Neither of these conflicts is new, and the Party has weathered flare-ups in both regions before. What is new is their concurrence with each other and with a number of other serious political and economic challenges. As the conflicts in Xinjiang and Hong Kong intensify, China’s real-estate market finally appears to be cooling, with potentially significant effects on the country’s economy, and pollution remains a national crisis that continues to stir sporadic unrest among otherwise “ordinary” citizens. And, of course, Party leaders are simultaneously pursuing an anti-corruption campaign that is hitting higher and higher targets. This campaign is ostensibly intended to bolster the economy and to address popular frustration over abuses of power, but like any purge, it also risks generating fresh enemies.

For reasons Barbara Geddes helps to illuminate (here), consolidated single-party authoritarian regimes like China’s tend to be quite resilient. They persist because they usually do a good job suppressing domestic opponents and co-opting would-be rivals within the ruling party. Single-party regimes are better than others at co-opting internal rivals because, under all but exceptional circumstances, regime survival reliably generates better payoffs for all factions than the alternatives.

Eventually, though, even single-party regimes break down, and when they do, it’s usually in the face of an economic crisis that simultaneously stirs popular frustration and weakens incentives for elites to remain loyal (on this point, see Haggard and Kaufman, too). Exactly how these regimes come undone is a matter of local circumstance and historical accident, but generally speaking, the likelihood increases as popular agitation swells and the array of potential elite defectors widens.

China’s slowing growth rate and snowballing financial troubles indicate that the risk of an economic crisis is still increasing. At the same time, the crises in Hong Kong, Xinjiang, and the many cities and towns where citizens are repeatedly protesting against pollution and corruption suggest that insiders who choose to defect would have plenty of potential allies to choose from. As I’ve said before, I don’t believe that the CPC regime is on the brink of collapse, but I would be surprised to see it survive in its current form—with no legal opposition and direct elections in rural villages only—to and through the Party’s next National Congress, due in in 2017.

Scientific Updating as a Social Process

Cognitive theories predict that even experts cope with the complexities and ambiguities of world politics by resorting to theory-driven heuristics that allow them: (a) to make confident counterfactual inferences about what would have happened had history gone down a different path (plausible pasts); (b) to generate predictions about what might yet happen (probable futures); (c) to defend both counterfactual beliefs and conditional forecasts from potentially disconfirming data. An interrelated series of studies test these predictions by assessing correlations between ideological world view and beliefs about counterfactual histories (Studies 1 and 2), experimentally manipulating the results of hypothetical archival discoveries bearing on those counterfactual beliefs (Studies 3-5), and by exploring experts’ reactions to the confirmation or disconfirmation of conditional forecasts (Studies 6-12). The results revealed that experts neutralize dissonant data and preserve confidence in their prior assessments by resorting to a complex battery of belief-system defenses that, epistemologically defensible or not, make learning from history a slow process and defections from theoretical camps a rarity.

That’s the abstract to a 1999 AJPS paper by Phil Tetlock (emphasis added; ungated PDF here). Or, as Phil writes in the body of the paper,

The three sets of studies underscore how easy it is even for sophisticated professionals to slip into borderline tautological patterns of thinking about complex path-dependent systems that unfold once and only once. The risk of circularity is particularly pronounced when we examine reasoning about ideologically charged historical counterfactuals.

As noted in a recent post, ongoing debates over who “lost” Iraq or how direct U.S. military intervention in Syria might or might not have prevented wider war in the Middle East are current cases in point.

This morning, though, I’m intrigued by Phil’s point about the rarity of defections from theoretical camps tied to wider belief systems. If that’s right—and I have no reason to doubt that it is—then we should not put much faith in any one expert’s ability to update his or her scientific understanding in response to new information. In other words, we shouldn’t expect science to happen at the level of the individual. Instead, we should look wherever possible at the distribution of beliefs across a community of experts and hope that social cognition is more powerful than our individual minds are.

This evidence should also affect our thinking about how scientific change occurs. In The Structure of Scientific Revolutions, Thomas Kuhn (p. 19 in the 2nd Edition) described scientific revolutions as a process that happens at both the individual and social levels:

When, in the development of a natural science, an individual or group first produces a synthesis able to attract most of the next generation’s practitioners, the older schools gradually disappear. In part their disappearance is caused by their members’ conversion to the new paradigm. But there are always some men who cling to one or another of the older views, and they are simply read out of the profession, which thereafter ignores their work.

If I’m reading Tetlock’s paper right, though, then this description is only partly correct. In reality, scientists who are personally and professionally (or cognitively and emotionally?) invested in existing theories probably don’t convert to new ones very often. Instead, the recruitment mechanism Kuhn also mentions is probably the more relevant one. If we could reliably measure it, the churn rate associated with specific belief clusters would be a fascinating thing to watch.

Retooling

Over the next year, I plan to learn how to write code to do text mining.

I’m saying this out loud for two reasons. The first is self-centered; I see a public statement about my plans as a commitment device. By saying publicly that I plan to do this thing, I invest some of my credibility in following through, and my credibility is personally and professionally valuable to me.

I’m also saying this out loud, though, because I believe that the thinking behind this decision might interest other people working in my field. There are plenty of things I don’t know how to do that would be useful in my work on understanding and forecasting various forms of political instability. Three others that spring to mind are Bayesian data analysis, network theory, and agent-based modeling.

I’m choosing to focus on text mining instead of something else because I think that the single most significant obstacle to better empirical analysis in the social sciences is the scarcity of data, and I think that text mining is the most promising way out of this desert.

The volume of written and recorded text we produce on topics of interest to social scientists is incomprehensibly vast. Advances in computing technology and the growth of the World Wide Web have finally made it possible to access and analyze those texts—contemporary and historical—on a large scale with efficiency. This situation is still new, however, so most of this potential remains unrealized. There is a lot of unexplored territory on the other side of this frontier, and that territory is still growing faster than our ability to map it.

Lots of other people in political science and sociology are already doing text mining, and many of them are probably doing it better than I ever will.  One option would be to wait for their data sets to arrive and then work with them.

My own restlessness discourages me from following that strategy, but there’s also a principled reason not just to take what’s given: we do better analysis when we deeply understand where our data come from. The data sets you know the best are the ones you make. The data sets you know second-best are the ones someone else made with a process or instruments you’ve also used and understand. Either way, it behooves me to learn what these instruments are and how to apply them.

Instead of learning text mining, I could invest my time in learning other modeling and machine-learning techniques to analyze available data. My modeling repertoire is pretty narrow, and the array of options is only growing, so there’s plenty of room for improvement on that front, too.

In my experience, though, more complex models rarely add much to the inferential or predictive power we get from applying relatively simple models to the right data. This may not be true in every field, but it tends to be true in work on political stability and change, where the phenomena are so complex and still so poorly understood. On these topics, the best we can usually do is to find gross patterns that recur among data representing theoretically coherent processes or concepts.

Relatively simple models usually suffice to discover those gross patterns. What’s harder to come by are the requisite data. I think text mining is the most promising way to make them, so I am now going to learn how to do it.

Follow

Get every new post delivered to your Inbox.

Join 6,496 other followers

%d bloggers like this: