Why Skeptics Make Bad Pundits

First Rule of Punditry: I know everything; nothing is complicated.

First Rule of Skepticism: I know nothing; everything is complicated.

Me on BuzzFeed on Venezuela

Journalist Rosie Gray has a story up at BuzzFeed on the wave of protests occurring now in Venezuela and the backdrop of economic crisis and political polarization against which it’s occurring. I found the piece interesting and informative, but I think it also illustrates how hard it is for journalists—and, for that matter, social scientists—to avoid openly sympathizing with one “side” or another in their reporting on conflicts like Venezuela’s and thereby leading readers to do the same.

Analytically, Gray’s piece attempts to explain why this wave of protests is occurring now and why anti-government activists have largely failed so far, in spite of the country’s severe economic problems, to draw large numbers of government supporters to their cause. Most of the sources quoted in Gray’s story are opposition activists, and they are generally described sympathetically. The first opposition activist we encounter, Carlos Vargas, tells us that he and other student protesters are “making an effort to reach out to the poor.” The next, a community organizer, admits that the opposition hasn’t made serious efforts to organize in his neighborhood, but we are then reminded that censorship and pro-government paramilitaries make it very hard for them to do so.

Gray also includes portions of an interview with two Chavistas, members of a colectivo in the 23 de Enero neighborhood. This interview and one with a pro-government economist ostensibly provide the “balance” in the piece, but their remarks and other descriptions of activity sympathetic to the government are framed in a way that evokes a sense of false consciousness. Hugo Chavez is dead, but he remains popular because of a “personality cult” that “still holds a grip on many Venezeulans, especially the poor.” Gray reports the government’s line that anti-government protesters “are a group of revanchist elites out of touch with regular Venezuelans” and writes that this line has “some grain of truth.” She immediately follows that sentence, however, with a description of protesters’ efforts to recruit poorer Venezuelans who, we are told by two of Gray’s sources, would participate more if they weren’t being menaced by pro-government militias. Gray tells us that the Chavistas she interviewed in 23 de Enero have a picture of Syrian president Bashar al-Assad on their wall, and that they blame their country’s unrest on “right-wing elements” in the U.S. and some of its allies. As for where ideas like that one come from, we are told that

Across town, the Chavista intelligentsia is hard at work coming up with theories for the foot soldiers to buy into.

To me, all of those phrases and details convey a belief that Chavistas aren’t joining the protesters because they are being duped. As a social scientist, I find that hypothesis unconvincing. The model of political behavior it implies echoes some instrumentalist theories of ethnic conflict, which posit that ethnic groups fight each other because self-interested leaders goad them into doing so. Those leaders’ efforts are certainly relevant to the story, but simple versions of the theory beg the question of why anyone listens. To try to understand that, we need more sympathetic accounts of the beliefs and choices made by those ostensible followers. Gray’s piece suggests one answer to that question when she recounts protesters’ claims that Chavista militias are intimidating them into obedience, but that also seems like a partial explanation at best. After all, some people are protesting in spite of that intimidation, so why not others?

This slant matters because it affects our judgments about what is possible and what is right, and those judgments affect the actions we and our governments take. Objectivity is an impossible ideal, not just for reporters but for anyone. Still, I think political reporters should aspire to afford the same sympathy to all of their sources and the causes they espouse, and then trust their readers to draw their own conclusions. Measured against that standard, I think Gray’s Venezuela piece—and, frankly, much of the reporting we get on factional disputes and popular protest in all parts of the world—fell a bit short.

How Circumspect Should Quantitative Forecasters Be?

Yesterday, I participated in a panel discussion on the use of technology to prevent and document mass atrocities as part of an event at American University’s Washington College of Law to commemorate the Rwandan genocide.* In my prepared remarks, I talked about the atrocities early-warning system I’m helping build for the U.S. Holocaust Memorial Museum’s Center for the Prevention of Genocide. The chief outputs of that system are probabilistic forecasts, some from statistical models and others from a “wisdom of (expert) crowds” system called an opinion pool.

After I’d described that project, one of the other panelists, Patrick Ball, executive director of Human Rights Data Analysis Group, had this to say via Google Hangout:

As someone who uses machine learning to build statistical models—that’s what I do all day long, that’s my job—I’m very skeptical that models about conflict, about highly rare events that have very complicated and situation-unique antecedents are forecastable. I worry about early warning because when we build models we listen to people less. I know that, from my work with the U.N., when we have a room full of people who know an awful lot about what’s going on on the ground, a graph—when someone puts a graph on the table, everybody stops thinking. They just look at the graph. And that worries me a lot.

In 1994, human-rights experts warned the world about what was happening [in Rwanda]. No one listened. So as we, as technologists and people who like technology, when we ask questions of data, we have to make sure that if anybody is going to listen to us, we’d better be giving them the right answers.

Maybe I was being vain, but I heard that part of Patrick’s remarks as a rebuke of our early-warning project and pretty much every other algorithm-driven atrocities and conflict forecasting endeavor out there. I responded by acknowledging that our forecasts are far from perfect, but I also asserted that we have reason to believe they will usually be at least marginally better than the status quo, so they’re worth doing and sharing anyway.

A few minutes later, Patrick came back with this:

When we build technology for human rights, I think we need to be somewhat thoughtful about how our less technical colleagues are going to hear the things that we say. In a lot of meetings over a lot of years, I’ve listened to very sophisticated, thoughtful legal, qualitative, ethnographic arguments about very specific events occurring on the ground. But almost inevitably, when someone proposes some kind of quantitative analysis, all that thoughtful reasoning escapes the room… The practical effect of introducing any kind of quantitative argument is that it displaces the other arguments that are on the table. And we are naive to think otherwise.

What that means is that the stakes for getting these kinds of claims right are very high. If we make quantitative claims and we’re wrong—because our sampling foundations are weak, because our model is inappropriate, because we misinterpreted the error around our claim, or for any other reason—we can do a lot of harm.

From that combination of uncertainty and the possibility for harm, Patrick concludes that quantitative forecasters have a special responsibility to be circumspect in the presentation of their work:

I propose that one of the foundations of any kind of quantitative claims-making is that we need to have very strict validation before we propose a conclusion to be used by our broader community. There are all kinds of rules about validation in model-building. We know a lot about it. We have a lot of contexts in which we have ground truth. We have a lot of historical detail. Some of that historical detail is itself beset by these sampling problems, but we have opportunities to do validation. And I think that any argument, any claim that we make—especially to non-technical audiences—should lead with that validation rather than leaving it to the technical detail. By avoiding discussing the technical problems in front of non-technical audiences, we’re hiding stuff that might not be working. So I warn us all to be much stricter.

Patrick has applied statistical methods to human-rights matters for a long time, and his combined understanding of the statistics and the advocacy issues is as good if not better than almost anyone else’s. Still, what he described about how people respond to quantitative arguments is pretty much the exact opposite of my experience over 15 years of working on statistical forecasts of various forms of political violence and change. Many of the audiences to which I’ve presented that work have been deeply skeptical of efforts to forecast political behavior. Like Patrick, many listeners have asserted that politics is fundamentally unquantifiable and unpredictable. Statistical forecasts in particular are often derided for connoting a level of precision that’s impossible to achieve and for being too far removed from the messy reality of specific places to produce useful information. Even in cases where we can demonstrate that the models are pretty good at distinguishing high-risk cases from low-risk ones, that evidence usually fails to persuade many listeners, who appear to reject the work on principle.

I hear loud echoes of my experiences in Daniel Kahneman’s discussion of clinical psychologists’ hostility to algorithms and enduring prejudice in favor of clinical judgment, even in situations where the former is demonstrably superior to the latter. On pp. 228 of Thinking, Fast and Slow, Kahneman observes that this prejudice “is an attitude we can all recognize.”

When a human competes with a machine, whether it is John Henry a-hammerin’ on the mountain or the chess genius Garry Kasparov facing off against the computer Deep Blue, our sympathies lie with our fellow human. The aversion to algorithms making decisions that affect humans is rooted in the strong preference that many people have for the natural over the synthetic or artificial.

Kahneman further reports that

The prejudice against algorithms is magnified when the decisions are consequential. [Psychologist Paul] Meehl remarked, ‘I do not quite know how to alleviate the horror some clinicians seem to experience when they envisage a treatable case being denied treatment because a ‘blind, mechanical’ equation misclassifies him.’ In contrast, Meehl and other proponents of algorithms have argued strongly that it is unethical to rely on intuitive judgments for important decisions if an algorithm is available that will make fewer mistakes. Their rational argument is compelling, but it runs against a stubborn psychological reality: for most people, the cause of a mistake matters. The story of a child dying because an algorithm made a mistake is more poignant than the story of the same tragedy occurring as a result of human error, and the difference in emotional intensity is readily translated into a moral preference.

If our distaste for algorithms is more emotional than rational, then why do forecasters who use them have a special obligation, as Patrick asserts, to lead presentations of their work with a discussion of the “technical problems” when experts offering intuitive judgments almost never do? I’m uncomfortable with that requirement, because I think it unfairly handicaps algorithmic forecasts in what is, frankly, a competition for attention against approaches that are often demonstrably less reliable but also have real-world consequences. This isn’t a choice between action or inaction; it’s a trolley problem. Plenty of harm is already happening on the current track, and better forecasts could help reduce that harm. Under these circumstances, I think we behave ethically when we encourage the use of our forecasts in honest but persuasive ways.

If we could choose between forecasting and not forecasting, then I would be happier to set a high bar for predictive claims-making and let the validation to which Patrick alluded determine whether or not we’re going to try forecasting at all. Unfortunately, that’s not the world we inhabit. Instead, we live in a world in which governments and other organizations are constantly making plans, and those plans incorporate beliefs about future states of the world.

Conventionally, those beliefs are heavily influenced by the judgments of a small number of experts elicited in unstructured ways. That approach probably works fine in some fields, but geopolitics is not one of them. In this arena, statistical models and carefully designed procedures for eliciting and combining expert judgments will also produce forecasts that are uncertain and imperfect, but those algorithm-driven forecasts will usually be more accurate than the conventional approach of querying one or a few experts and blending their views in our heads (see here and here for some relevant evidence).

We also know that most of those subject-matter experts don’t abide by the rules Patrick proposes for quantitative forecasters. Anyone who’s ever watched cable news or read an op-ed—or, for that matter, attended a panel discussion—knows that experts often convey their judgments with little or no discussion of their cognitive biases and sources of uncertainty.

As it happens, that confidence is persuasive. As Kahneman writes (p. 263),

Experts who acknowledge the full extent of their ignorance may expect to be replaced by more confident competitors who are better able to gain the trust of clients. An unbiased appreciation of uncertainty is a cornerstone of rationality—but it is not what people and organizations want. Extreme uncertainty is paralyzing under dangerous circumstances, and the admission that one is merely guessing is especially unacceptable when the stakes are high. Acting on pretended knowledge is often the preferred solution.

The allure of confidence is dysfunctional in many analytic contexts, but it’s also not something we can wish away. And if confidence often trumps content, then I think we do our work and our audiences a disservice when we hem and haw about the validity of our forecasts as long as the other guys don’t. Instead, I believe we are behaving ethically when we present imperfect but carefully derived forecasts in a confident manner. We should be transparent about the limitations of the data and methods, and we should assess the accuracy of our forecasts and share what we learn. Until we all agree to play by the same rules, though, I don’t think quantitative forecasters have a special obligation to lead with the limitations of their work, thus conceding a persuasive advantage to intuitive forecasters who will fill that space and whose prognostications we can expect to be less reliable than ours.

* You can replay a webcast of that event here. Our panel runs from 1:00:00 to 2:47:00.

The “Cuban Twitter” Fiasco

The Associated Press dropped a big investigative story this morning on how “The U.S. government masterminded the creation of a ‘Cuban Twitter’—a communications network designed to undermine the communist government in Cuba, built with secret shell companies and financed through foreign banks.”

The project, which lasted more than two years and drew tens of thousands of subscribers, sought to evade Cuba’s stranglehold on the Internet with a primitive social media platform. First, the network would build a Cuban audience, mostly young people; then, the plan was to push them toward dissent.

Yet its users were neither aware it was created by a U.S. agency with ties to the State Department, nor that American contractors were gathering personal data about them, in the hope that the information might be used someday for political purposes.

It is unclear whether the scheme was legal under U.S. law, which requires written authorization of covert action by the president and congressional notification. Officials at USAID would not say who had approved the program or whether the White House was aware of it. The Cuban government declined a request for comment.

If you study or work on democratization or development, this story is one you’ve got to read. That “U.S. agency with ties to the State Department” mentioned in the snippet above is none other than USAID, the supposedly benign and benevolent arm of U.S. development assistance around the world.

As I read the story, I kept thinking: how myopic. I don’t have time this morning to write a post explaining why I think this is a terrible idea, so I hope you’ll forgive me for quoting from a post I wrote on the same topic nearly three years ago, when talk of U.S. government–funded “Internet in a suitcase” programs aimed at keeping the Arab Spring rolling was hotting up. I hope you’ll take a minute to read the whole thing, but here’s my core complaint:

What worries me is that those well-intentioned officials may not have thought through how modest support for activists in authoritarian regimes might backfire. In a paper I presented at an academic conference a few years ago, I used game theory to explore the conditions under which authoritarian rulers might expand civil liberties in spite of the attendant threats to their power in an effort to reduce government expenses and accelerate economic growth. The formal model in that paper suggested that, other things being equal, autocrats are most likely to liberalize when political opponents pose either a grave threat or a minimal threat to their power. When would-be rivals pose a moderate threat, autocrats will feel compelled to keep the screws turned tight to prevent those rivals from gaining the strength that could transform them into a formidable foe. In this situation, the risks of liberalization will often outweigh the potential benefits.

If foreign governments reliably provided enough support to budding opposition movements to enable those movements to overwhelm autocrats’ defenses, we might expect injections of foreign support to spur autocrats to liberalize before they get toppled. As long as foreign contributions fall short of those heights, however—and they nearly always do—autocrats have strong incentives to respond to those interventions by clamping down, not opening up. This problem may be exacerbated by a substitution effect, whereby activists choose to invest less of their own time and money in overcoming barriers to communication because they expect foreign interventions to solve those problems. In other words, the chief outcomes we would expect to see from foreign support for popular uprisings would be more repression, not less, and weaker prospects for a transition to democracy.

The other issue I touched on at the end that post was the effect a revelation like this one has on USAID’s other endeavors, many of which of which are fairly straightforward and well-intentioned programs aimed at improving peoples’ lives in more fundamental ways, like vaccination and nutrition. Programs like this “Cuban Twitter” fiasco erode USAID’s credibility as an agent of development assistance everywhere. “If the U.S. government used USAID as a Trojan horse in Cuba,” politicians around the world might ask themselves, “why not in my country, too?” It’s hard for me to see whatever marginal effect this Cuban program might have had on the prospects for regime change in that country being worth the costs those doubts will impose on USAID’s work everywhere else.

Reform in Burma Isn’t Unraveling (Yet), But Our Narrative About It Sure Is

If a couple of recent pieces on Foreign Policy‘s website are to be believed, the democratization process that sputtered to life in Burma two and a half years ago has stalled and is now rolling back downhill. In “Hillary’s Burma Problem,” Catherine Traywick and John Hudson argue that “the promise of a free and democratic Myanmar is rapidly receding as sectarian violence escalates and the government backslides on a number of past reforms.” Meanwhile, Democracy Lab blogger Min Zin tells us that, for the past few months, he’s been “unable to escape an ominous sense that the political situation in Burma is on the wrong track,” and he points to a leadership crisis and a growing risk of social unrest as the chief sources of his anxiety.

I won’t dispute any of the facts in those pieces, and I’ve been an avid reader of Min Zin’s excellent Democracy Lab posts as long as he’s been writing them. As I argued on this blog a couple of years ago, though, I think it’s more accurate to think of what’s happened in Burma so far not as a transition to democracy, but as a case of liberalization from above that may or may not produce a try at democratic government in the next few years.

Is that a distinction without a difference? I don’t think so. As O’Donnell and Schmitter propose in their Little Green Book, liberalization involves the expansion of freedoms from arbitrary acts of the state and others, while democratization entails the expansion of popular consultation and accountability. The two processes often coincide, but they are usefully construed as distinct streams of political change. Crucially, while democratic government is impossible without civil liberties—especially freedoms of speech, association, and assembly—liberalization can and sometimes does occur without any democratization.

Understood on those terms, I think the liberalization process in Burma has progressed incrementally but significantly in the past two years and has not yet regressed in any substantial way, with the partial but significant exception of the plight of the Rohingya. What Burma’s liberalization has done is create space for new political and economic activity, and as is often the case, not all of what people are doing with that space is progressive or good. On the positive side of the ledger, freedoms of speech and the press remain incomplete but are much improved. Political prisoners have been released and not restocked. Apparently, there’s even a budding startup scene in Yangon. On the negative side of the ledger, the prospect of new fortunes is spurring land grabs by elites, and attempts to protest those displacements and the pollution that sometimes follow have largely been ignored or harshly repressed. And, of course, some Burmans have responded to the opening by mobilizing around an aggressive chauvinism that has already produced what amounts to a slow-rolling episode of ethnic cleansing and still threatens to slide into genocide.

As is sometimes but not always the case, this partial liberalization has also been accompanied by some significant but still limited elements of democratization, too. Parliamentary by-elections were held in 2012, opposition parties won nearly all the seats at stake, and no one shut the process down. More recently, word came that the National League for Democracy, the party of ostensible opposition leader Aung Saan Suu Kyi, would field a candidate for president in balloting scheduled for next year, even if Suu Kyi herself is not permitted to run.

What we still haven’t seen, though, is any clear sign that deeply entrenched elites plan to allow that process to threaten their station. Rather, what’s emerged so far is more like the arrangements that hold in monarchies like Morocco or Jordan. There, loyal opposition parties are allowed to contest seats in the legislature, and a certain amount of free discourse and even protest is tolerated, but formal and informal rules ensure that incumbent insiders retain control over the political agenda and veto power over all major decisions.

For that to change in Burma, the country’s constitution would have to change. When military elites rewrote that document a few years ago, however, they cleverly ensured that constitutional reform couldn’t happen without their approval. So far, we have seen no signs that they plan to relinquish that arrangement any time soon. Until we do, I think it’s premature to speak of a transition to democracy in Burma. Democratization, yes, but not enough yet to say that the country is between political orders. What we have now, I think, is a partially liberalized authoritarian regime that’s still led by a military elite with uncertain intentions.

To make sure this view wasn’t crazy, I queried Brian Joseph, senior director for Asia and Global Programs at the National Endowment for Democracy and a longtime Burma watcher who also happens to be the father of one of my son’s classmates. In particular, I asked Brian by email if he agreed with Traywick and Hudson’s thesis that the “transition” in Burma was “unraveling.” He pointed me toward Min Zin’s piece as “a more informative analysis” and said he agreed with Min Zin that “the transition’s trajectory is no longer clear” and then added parenthetically: “Not that I ever thought it was in the first place but that was clearly the message of the [international] community.”

Brian’s reference to “the message of the international community” in that aside is crucial to understanding how what I described here can be true and we can still see analyses claiming that Burma’s “transition” is “unraveling.” Best I can tell, what’s coming undone right now isn’t Burma’s reform process, although as Min Zin discusses, that certainly could happen, and there are plenty of reasons to fear that it might.

No, what I think we’re really seeing in articles like the one by Traywick and Hudson is an overdue deflation of the hype balloon Burma’s reforms have pumped up. With some help from various outsiders—some eager to see deeper political transformations occur, others looking to capitalize on the money-making opportunities this new market presents—we let our hopes for Burma’s future drive our narrative about what was happening in the present. The Arab Spring spurred a similar dynamic in American analysis of that part of the world. Let’s hope the whiplash over Burma isn’t as severe.

A Nice Pat on the Back

I had to leave the annual convention of the International Studies Association yesterday, before it wrapped up, but not before receiving a nice pat on the back. In the second annual Online Achievement in International Studies (OAIS) awards—a.k.a. the Duckies—Dart-Throwing Chimp was recognized as Best Blog (Individual).

It seems fitting to use this platform to thank the Duck of Minerva crew for organizing the OAIS awards and SAGE Publications for helping to make them happen. Most of all, though, I want to say thanks to all of you for reading and conversing with me. I hope I can keep it interesting.

Demography, Democracy, and Complexity

Five years ago, demographer Richard Cincotta claimed in a piece for Foreign Policy that a country’s age structure is a powerful predictor of its prospects for attempting and sustaining liberal democracy. “A country’s chances for meaningful democracy increase,” he wrote, “as its population ages.” Applying that superficially simple hypothesis to the data at hand, he ventured a forecast:

The first (and perhaps most surprising) region that promises a shift to liberal democracy is a cluster along Africa’s Mediterranean coast: Morocco, Algeria, Tunisia, Libya, and Egypt, none of which has experienced democracy in the recent past. The other area is in South America: Ecuador, Colombia, and Venezuela, each of which attained liberal democracy demographically “early” but was unable to sustain it. Interpreting these forecasts conservatively, we can expect there will be one, maybe two, in each group that will become stable democracies by 2020.

I read that article when it was published, and I recall being irritated by it. At the time, I had been studying democratization for more than 15 years and was building statistical models to forecast transitions to and from democracy as part of my paying job. Seen through those goggles, Cincotta’s construct struck me as simplistic to the point of naiveté. Democratization is a hard theoretical problem. States have arrived at and departed from democracy by many different pathways, so how could what amounts to a one-variable model possibly have anything useful to say about it?

Revisiting Cincotta’s work in 2014, I like it a lot more for a couple of reasons. First, I like the work better now because I have come to see it as an elegant representation of a larger idea. As Cincotta argues in that Foreign Policy article and another piece he published around the same time, demographic structure is one component of a much broader and more complex syndrome in which demography is both effect and cause. Changes in fertility rates, and through them age structure, are strongly shaped by other social changes like education and urbanization, which are correlated with, but hardly determined by, increases in national wealth.

Of course, that syndrome is what we conventionally call “development,” and the pattern Cincotta observes has a strong affinity with modernization theory. Cincotta’s innovation was to move the focus away from wealth, which has turned out to be unreliable as a driver and thus as a proxy for development in a larger sense, to demographic structure, which is arguably a more sensitive indicator of it. As I see it now, what we now call development is part of a “state shift” occurring in human society at the global level that drives and is reinforced by long-term trends in democratization and violent conflict. As in any complex system, though, the visible consequences of that state shift aren’t evenly distributed.

In this sense, Cincotta’s argument is similar to one I often find myself making about the value of using infant mortality rates instead of GDP per capita as a powerful summary measure in models of a country’s susceptibility to insurgency and civil war. The idea isn’t that dead children motivate people to attack their governments, although that may be one part of the story. Instead, the idea is that infant mortality usefully summarizes a number of other things that are all related to conflict risk. Among those things are the national wealth we can observe directly (if imperfectly) with GDP, but also the distribution of that wealth and the state’s will and ability to deliver basic social services to its citizens. Seen through this lens, higher-than-average infant mortality helps us identify states suffering from a broader syndrome that renders them especially susceptible to violent conflict.

Second, I have also come to appreciate more what Cincotta was and is doing because I respect his willingness to apply his model to generate and publish probabilistic forecasts in real time. In professional and practical terms, that’s not always easy for scholars to do, but doing it long enough to generate a real track record can yield valuable scientific dividends.

In this case, it doesn’t hurt that the predictions Cincotta made six years ago are looking pretty good right now, especially in contrast to the conventional wisdom of the late 2000s on the prospects for democratization in North Africa. None of the five states he lists there yet qualifies as a liberal democracy on his terms, a “free” designation from Freedom House). Still, it’s only 2014, one of them (Tunisia) has moved considerably in that direction, and two others (Egypt and Libya) have seen seemingly frozen political regimes crumble and substantial attempts at democratization ensue. Meanwhile, the long-dominant paradigm in comparative democratization would have left us watching for splits among ruling elites that really only happened in those places as their regimes collapsed, and many area experts were telling us in 2008 to expect more of the same in North Africa as far as the mind could see. Not bad for a “one-variable model.”

Today’s China Is Communist and Modern, Not High Modernist

This rebuttal to my recent post on China is a cross-post from Jeremy Wallace’s blog, Science of Politics, with his permission. You can see the original here. Thanks to a shout-out from Marginal Revolution, my post got a lot of views, and I hope Jeremy’s response will get the same. He’s the expert on China, so his argument has me reconsidering my views.

The Chinese government released its long-awaited urbanization plan (国家新型城镇化规划) on 16 March. Ian Johnson, who has written extensively about China’s urbanization for the New York Times, begins his piece on the announcement of the plan in grand terms:

China has announced a sweeping plan to manage the flow of rural residents into cities, promising to promote urbanization but also to solve some of the drastic side effects of this great uprooting.

These descriptions of nondemocratic regime’s releasing “sweeping” plans to reshape their economic geography made Jay Ulfelder think of High Modernism, largely from Jim Scott’sSeeing Like a State. Scott describes significant disasters that have emerged out of failed social engineering projects. Ulfelder quotes from a review of Scott’s excellent book by Cass Sunstein:

Scott does not deny that some designs are well-motivated, and he acknowledges that plans can sometimes do a lot of good. He is concerned to show that when a government, with its “thin simplifications” of complicated systems, fails to understand how human beings organize (and disorganize) themselves, its plans are doomed from the start. Scott calls some governments practitioners of “high modernism,” a recipe for many natural and social disasters, including tyranny… Left to itself, this ideology is overconfident but benign. [High modernism] becomes authoritarian when it is conjoined to “an authoritarian state that is willing and able to use the full weight of its coercive power to bring these high-modernist designs into being.” This is especially dangerous when it is linked to “a prostrate civil society that lacks the capacity to resist these plans.” Thus the greatest calamities in Scott’s book involve a weak society that cannot adapt to a government’s plans.

In some ways, then, the summary of the plan in the NYT looks like a classical example of High Modernism. As Ulfelder writes,

China’s sweeping plans for controlled urbanization strike me as high modernism par excellence. This scheme is arguably the twenty-first century version of agricultural collectivization—the kind of “revolution from above” that Stalin promised, only now the goal is to put people into cities instead of farms, and to harness market forces instead of refuting them. ”We are here on the path to modernity,” the thinking seems to go, “and we want to be there. We are a smart and powerful state, so we will meticulously plan this transformation, and then use our might to induce it.”

Such a characterization leads Ulfelder to two predictions.

If Scott is right about these “certain schemes,” though, then two things are liable to happen. First, China’s new plan for managed urbanization will probably fail on its own terms. It will fail because human planners don’t really understand how these processes work, and even if those planners did understand, they still couldn’t control them. This prediction doesn’t imply that China won’t continue to urbanize, or even that city-dwellers’ quality of life won’t continue to improve on average. It just means that those trends will continue in spite of these grand plans instead of because of them. If the American experience in Afghanistan—or, heck, in its own urban centers—is any guide, we should expect many of the housing developments, schools, and transportation infrastructure born of this plan to go underused and eventually to decay. Or, as an economist might put it, the return on investment will probably be poor.

The second prediction of sorts I take from Scott’s book is that the Chinese Communist Party’s plans for socially engineered urbanization will probably produce a lot of conflict and suffering on their way to failure.

I disagree with the assessment of the plan as high modernism and with the causal mechanisms underlying the predictions that arise from it. It isn’t high modernist because China doesn’t “plan” like it used to and the described policies incrementally adjust the status quo. The predictions themselves are not wrong so much as they are already correct.

First, the nature of planning in China has gradually moved away from the intense micro-managing of the eponymous Planned Economy to something much more akin to policies that shape the incentive structure of local governments and individuals by allocating marginal resources more to one locale rather than another. That is, China governs like a modern state, not a high modern one. Even the words used in plans have changed, as pointed out by Philipp C.C. Huang:

If one looks to the evolution in the Chinese terms for planning, we can see that the words have changed first from jihua 计划 and zhilingxing jihua 指令性计划 or “commandist planning” to zhidaoxing jihua 指导性计划 or “guidance planning,” and, more recently, to abandoning the old term jihua completely in favor of guihua 规划, now the commonly used term for what the new National Development and Reform Commission (国家发展和改革委员会), which replaced the old National Planning Commission (国家计划委员会), undertakes.

This semantic change reflects a real reduction in the Party’s control of the day-to-day operations of the economy. This can be seen in the fact that this document is often described as “long-awaited.” It is long-awaited because it was supposed to be announced last year. As Jamil Anderlini of the FT put it,

The urbanisation plan was originally expected to be published more than a year ago, but deep divisions between government departments and dissatisfaction from Li Keqiang, the Chinese premier, who has been a strong champion of the scheme, delayed the plan’s publication until now.

I would argue that this slowly rolled out plans like this one are less likely to be sweeping than those that emerge out of nowhere. Additionally, this dissatisfaction implies that, unlike in China under Mao, local implementation of the plan is unlikely to be anything but grudging. There is a growing literature on local resistance to implementing central dictates in China (e.g., Margaret Pearson and Mei Ciqi have a nice forthcoming paper in China Journal entitled “Killing the Chicken to Scare the Monkey: Sanctions, Shared Beliefs and Local Defiance in China” that I can’t find online).

Second, the document is not a radical departure from prior policy. Johnson’s statement “the plan [is] the country’s first attempt at broadly coordinating one of the greatest migrations in history” fits awkwardly with a history of policies regulating and restricting migration that have existed since 1950s (I might have just finished writing a book about China’s management of urbanization).

The household registration (hukou) system was established when Soviet-style industrialization was initiated to control that true high modernist policy’s unintended consequences, namely blind flows of farmers into cities looking for work and escaping rural taxation. This system of effective migration restrictions has been tinkered with at the national and subnational level countless times during China’s post-Mao Reform Era (1978–). Over the past ten years, such reforms have been constantly trumpeted but implemented reality rarely measures up to the hype of policy announcement. Yet reforms have certainly taken place; Tom Miller’s great China’s Urban Billion summarizes many recent changes well.

The newly released document describes policies that are broadly similar to what we have seen time after time in recent years: continued “strict control” of population growth in the largest cities and encouragement of development of small and medium-sized cities, particularly in the country’s central and western regions. What is different here is a central commitment to assist local government’s fund the infrastructure of their cities and efforts to contain “land urbanization,” where local governments claim rural land from village collectives, pay farmers a pittance, and sell it at a huge profit to developers. The urbanization of land causes the “forced urbanization” of individuals that Ian Johnson’s reporting decries, so attempts to reduce its prevalence going forward should be welcomed.

Why does this plan sound high modernist then? Because it emanates from a Communist Party-led regime that still tends to use language more appropriate to the grand pronouncements of Marxism. It is a Communist state. The regime retains the power to manage the economy and guides it towards in desired directions but in general refrains from stating desired ends.

As for the predictions coming from classifying China as high modernist, the country already is dealing with serious problems of ghost cities where any return on investment is questionable. It is certainly possible that aiding the development of small and medium cities will turn out being wasteful economically, even if it might be savvy politically. In terms of urban instability and violence, I’m sanguine. I see this plan as continuing in a long line of policies that the regime has put forward to try to avoid urban unrest–incorporating slums, expanding access to urban social services, and slowing down land confiscations–are all reasonable levers for the center to use to tamp down the possibilities of protest in cities.

In the end, the Chinese regime speaks with archaic language–that is indeed, occasionally frightening–but acts like a modern state. Today’s CCP leadership certainly prefers to depoliticize and to quantify, to argue that it is pursuing “development,” “progress,” and “modernization” without giving the Chinese people much of a voice to prevent them from doing so. But so do other modern states. China today is far from the catastrophes of its high modern era, namely the Great Leap Forward. Let us all be thankful that this is so.

On Prediction

It is surprisingly hard to find cogent statements about why prediction is important for developing theory in political science. This is primarily a cultural artifact, I think—political science has mostly eschewed prediction for decades, and many colleagues and reviewers remain openly hostile to it, so it’s not something we spend much time writing and talking about—but there’s a hard philosophy-of-science question lurking there, too. It’s the one Karl Popper implies when he argues in “Prediction and Prophecies in the Social Sciences” that:

Long-term prophecies can be derived from scientific conditional predictions only if they apply to systems which can be described as well-isolated, stationary, and recurrent. These systems are very rare in nature; and modern society is not one of them.

In other words, the causal processes social scientists aim to discover could be moving targets, so the theories we develop through our research will typically be bounded and contingent. If we’re not sure a priori how general we expect our theories to be, then how much predictive power should we expect them to have? To which cases is a theory meant to apply, and therefore to predict? Without an answer to that question, we can’t confidently judge how informative our predictive accuracy or errors are, and we can only answer that question with another layer of theory.

Statistical modeling suggests a practical rationale to prefer out-of-sample to in-sample “prediction” as means of validation, namely, overfitting. That’s what we call it when we build models that chase after the idiosyncrasies in the data on which they’re trained and lose sight along the way of the wider regularities we seek to uncover. As a technical matter, though, the problem of overfitting is specific to the method, and as philosophical point it’s another handwave. In particular, it doesn’t deal with the possibility that a theory works wonderfully for the sample from which it’s derived but poorly elsewhere. That proposition must sound bizarre to biologists or physicists, but as Popper and other would argue, it’s not a crazy idea in political science. Do we really think that the causes of war between states are the same now as they were 100 years ago? Of democratization? Human physiology may not evolve that fast, but human society arguably does, at least recently.

After all that hand-wringing, I land on a pragmatic rationale for emphasizing prediction as a means of validation: it works better than the alternatives. Science is a method for deepening our understanding of the world we inhabit. Deepening implies directional movement. For science to move, we have to try to assess the validity of, and adjudicate between, different ideas. From psychology, we know that the sheer plausibility of a story—and at some level, that’s really what all social-science theories are, whichever “language” we use to represent them—is not a reliable guide to its truth. Just because something makes sense or feels right does not mean that it is, and our brains are terrible about (or excellent at) filtering evidence to confirm our expectations.

Under those circumstances, we need some other way to assess whether an explanation is plausible—or, when more than one explanation is available, to determine which version is more plausible than the others. We can try to do that with reference to the evidence from which the explanation was derived, but the result is a tautology: evidence A suggests theory B; we know theory B is correct because it implies evidence A, and A is what we observe.

The alternative that remains is prediction. To determine whether or not our mental models are getting at something “real,” we have to apply them to situations yet unseen and see if the regularities we thought we had uncovered hold. The results will rarely be binary, but they will still provide new information about the usefulness of the model. Crucially, those results can also be compared to ones from competing theories to help us determine which explanation covers more. That’s the engine of accumulation. Under certain conditions, that new information may even reveal something about the scope conditions of the initial theory. When a model routinely predicts well in some kinds of cases but not others, then we have uncovered a new pattern that we can add to the initial construct and then continue to test in the same way.

For reasons to which Popper alludes, I don’t believe these iterations will reveal laws that consistently explain and anticipate political behavior writ large. Still, we have seen this process produce great advances in other fields, so I prefer it to the alternative of not trying. And, absent some version of this process, the theories we construct about politics are epistemologically indistinguishable from fiction. Fiction can be satisfying to write and read and even be “true” in a fashion, but it is not science because, among other things, it does not aspire to accumulation.

I use the word “aspire” in that last sentence advisedly. Directionality does not necessarily imply eventual arrival, or even reliable navigation. I think it’s perfectly reasonable to establish the accumulation of knowledge at the basic point of the endeavor and still understand and experience science as neuroscientist Stuart Firestein describes it in his wonderful book, Ignorance. Firestein opens the book with a proverb—”It is very difficult to find a black cat in a dark room, especially when there is no cat”—and goes on to say that

[Science] is not facts and rules. It’s black cats in dark rooms. As the Princeton mathematician Andrew Wiles describes it: It’s groping and probing and poking, and some bumbling and bungling, and then a switch is discovered, often by accident, and the light is lit, and everyone says, “Oh, wow, so that’s how it looks,” and then it’s off into the next dark room, looking for the next mysterious black feline. If this all sounds depressing, perhaps some bleak Beckett-like scenario of existential endlessness, it’s not. In fact, it’s somehow exhilarating.

From my own experience, I’d say it can be both depressing and exhilarating, but the point still stands.

China Isn’t Socialist, It’s High Modernist

In today’s New York Times, Ian Johnson reports that

China has announced a sweeping plan to manage the flow of rural residents into cities, promising to promote urbanization but also to solve some of the drastic side effects of this great uprooting…

[The plan] states that “urbanization is modernization” and “urbanization is an inevitable requirement for promoting social progress,” noting that every developed country is urbanized and industrialized.

In certain circles of development studies, it’s become almost cliché to invoke James Scott’s Seeing Like a State: How Certain Schemes to Improve the Human Condition Have FailedI’m going to do it anyway—because the book is that good, but also because Scott’s framework suggests two important predictions about where China’s process of managed urbanization is headed.

For a quick synopsis of Scott’s masterwork, I’ll turn to a 1998 review of it by Cass Sunstein. Sunstein describes Scott’s book as a study of social engineering, or “selective interventions into complex systems,” and the moral of the story is that these interventions rarely end well.

Scott does not deny that some designs are well-motivated, and he acknowledges that plans can sometimes do a lot of good. He is concerned to show that when a government, with its “thin simplifications” of complicated systems, fails to understand how human beings organize (and disorganize) themselves, its plans are doomed from the start. Scott calls some governments practitioners of “high modernism,” a recipe for many natural and social disasters, including tyranny… Left to itself, this ideology is overconfident but benign. [High modernism] becomes authoritarian when it is conjoined to “an authoritarian state that is willing and able to use the full weight of its coercive power to bring these high-modernist designs into being.” This is especially dangerous when it is linked to “a prostrate civil society that lacks the capacity to resist these plans.” Thus the greatest calamities in Scott’s book involve a weak society that cannot adapt to a government’s plans.

The intellectual core of Scott’s book is a theory of incremental state-building, but its moral core is a set of observations about cases where high modernist ideology and authoritarian states have come together to produce especially disastrous social outcomes.

So what is this ideology? As Scott explains (pp. 89-90), high modernism

is best conceived as a strong (one might even say muscle-bound) version of the beliefs in scientific and technical progress that were associated with industrialization in Western Europe and in North America from roughly 1830 until World War I. At its center was a supreme self-confidence about continued linear progress, the development of scientific and technical knowledge, the expansion of production, the rational design of social order, the growing satisfaction of human needs, and, not least, an increasing control over nature (including human nature) commensurate with scientific understanding of natural laws. High modernism is thus a particularly sweeping vision of how the benefits of technical and scientific progress might be applied—usually through the state—in every field of human activity… The high-modernist state began with extensive prescriptions for a new society, and it intended to impose them.

High modernism was on full display in many of the USSR’s grand developmental schemes, from the agricultural collectivization drives that killed millions to the massive river diversion project that was finally abandoned in 1986. High modernism has also afflicted Western state-building efforts in Afghanistan (here), and those efforts have often foundered in the very ways that Scott’s book anticipates (here).

China’s sweeping plans for controlled urbanization strike me as high modernism par excellence. This scheme is arguably the twenty-first century version of agricultural collectivization—the kind of “revolution from above” that Stalin promised, only now the goal is to put people into cities instead of farms, and to harness market forces instead of refuting them. “We are here on the path to modernity,” the thinking seems to go, “and we want to be there. We are a smart and powerful state, so we will meticulously plan this transformation, and then use our might to induce it.”

If Scott is right about these “certain schemes,” though, then two things are liable to happen. First, China’s new plan for managed urbanization will probably fail on its own terms. It will fail because human planners don’t really understand how these processes work, and even if those planners did understand, they still couldn’t control them. This prediction doesn’t imply that China won’t continue to urbanize, or even that city-dwellers’ quality of life won’t continue to improve on average. It just means that those trends will continue in spite of these grand plans instead of because of them. If the American experience in Afghanistan—or, heck, in its own urban centers—is any guide, we should expect many of the housing developments, schools, and transportation infrastructure born of this plan to go underused and eventually to decay. Or, as an economist might put it, the return on investment will probably be poor.

The second prediction of sorts I take from Scott’s book is that the Chinese Communist Party’s plans for socially engineered urbanization will probably produce a lot of conflict and suffering on their way to failure. The capacity of Chinese civil society to resist these schemes is not great, but it also varies a great deal across issues and locales and appears to be strengthening. We see hints of this resistance and its coming intensification in Johnson’s story:

Separately, state television reported on Sunday night that 4.75 million people living in shantytowns would have their housing improved this year. These areas are often villages that have been swallowed up by cities, and at times have been flashpoints of violence between municipal officials who want to demolish them and residents unwilling to move. It is unclear whether the plan will significantly raise relocation compensation for the residents of these areas.

Now, I can think of at least two ways these predictions might not come true. First, the CPC might not really try to implement this plan, or it might abandon the plan if and when conflict arises. I have a hard time imagining that outcome, though, precisely because the Party has now become so publicly invested in high modernist ideology. The Party’s claim to public authority is now lashed to the idea of it as a benevolent and capable modernizer, so any obvious slackening of that commitment would open the door to conflict over what or who should replace it.

Second, these predictions might not come true because the Chinese Communist Party might succeed where all others have failed. So, has the Chinese Communist Party cracked the code on “how human beings organize (and disorganize) themselves”, as Sunstein put it? And has it married that never-before-achieved understanding with an unprecedented capacity for design and implementation? If you don’t say yes to both of those questions, it’s hard to see how this scheme manages to pull off what no other comparable scheme before it has done.

Follow

Get every new post delivered to your Inbox.

Join 5,705 other followers

%d bloggers like this: