Statistics Is Not Alchemy

Are aid and investment from China driving crackdowns on the press in some parts of Africa?

I don’t know.

That’s unsatisfying and maybe even a little annoying, but I’m writing a post about it anyway because why I don’t know says a lot about how hard it is to do good quantitative social science, even in the age of Big Data. Here’s the story:

A few Mondays ago, the New York Times ran an op-ed entitled “Africa’s Free Press Problem” in which the author, Mohamed Keita of the advocacy group Committee to Protect Journalists, asserted that press freedom is eroding in Africa, and foreign forces are partially at fault. According to Keita, “Independent African journalists covering the continent’s development are now frequently persecuted for critical reporting on the misuse of public finances, corruption and the activities of foreign investors.” He lays part of the blame for this alleged trend at the feet of Western governments more interested in promoting economic development and stability than democracy, but he sees other forces at work, too:

Then there’s the influence of China, which surpassed the West as Africa’s largest trading partner in 2009. Ever since, China has been deepening technical and media ties with African governments to counter the kind of critical press coverage that both parties demonize as neocolonialist.

In January, Beijing issued a white paper calling for accelerated expansion of China’s news media abroad and the deployment of a press corps of 100,000 around the world, particularly in priority regions like Africa. In the last few months alone, China established its first TV news hub in Kenya and a print publication in South Africa. The state-run Xinhua news agency already operates more than 20 bureaus in Africa. More than 200 African government press officers received Chinese training between 2004 and 2011 in order to produce what the Communist Party propaganda chief, Li Changchun, called “truthful” coverage of development fueled by China’s activities.

When I finished Keita’s piece, I was sympathetic to his concerns, but I was skeptical of his claim that the ebb and flow of press freedom in Africa was being shaped so decisively by China’s recent investments on the continent. From my own reading of politics, I see the kinds of constraints on the press that Keita describes in Ethiopia and Rwanda as normal features of authoritarian rule. By my reckoning, both Ethiopia and Rwanda have been repressing independent journalism for quite a while, so I couldn’t see how China’s recent overtures would have much to do with why that repression is happening. Cause has to precede effect and all that.

Being an empiricist and a blogger, I figured I’d pursue my hunch by taking a look at the data and writing a post. In a day or two, I could run a statistical analysis that would check Keita’s implied claim that Chinese engagement was reducing press freedom in Africa. I knew that both Freedom House and Reporters Without Borders produce annual, country-level measures of press freedom covering at least the past decade, so I was confident that I could observe recent trends on that side of the equation. All I needed was comparable data on aid and foreign direct investment from China, and I could run some simple fixed-effects models to see if changes over time in those money flows really were associated with decreases in press freedom, as Keita’s essay seemed to suggest.

And that’s where I hit a wall. First, I Googled “china foreign investment data” and “china foreign aid data” and came up with next to the nothing. The best I could do was an incomplete, project-level data set of Chinese foreign aid projects in Africa from 1990 through 2005. Next, I posted queries on Twitter and the listserv of the Society for Political Methodology. The latter led me to the University of the Pacific’s Daniel O’Neill, who confirmed my growing suspicion that the data I wanted simply don’t exist. We can see annual outflows of FDI from China, but we can’t see where that money’s gone, and bilateral data on development assistance from China are not available. (Even if they were, I’m not sure I would have trusted the numbers, but that’s beside the point for now.)

So, here we are in 2012, and it’s impossible to answer a seemingly simple question because the data we need to answer that question are nowhere to be found.

In fact, there are a lot of really interesting and important social-science questions where this is true. Income inequality is one of them, as I discussed on this blog a few weeks ago. Unemployment is another. If I had a dollar for every time I heard someone suggested adding unemployment to a global statistical model of political instability, I’d be a lot richer. It turns out, though, that many countries don’t report unemployment rates, and many of the ones that do only started to do so recently. A quick look at the World Bank’s World Development Indicators shows the problem clearly; lots of countries have no observations, and those gaps are correlated with other things that contribute to the risks of political instability–poverty most especially, but also authoritarian rule and recent or ongoing civil violence.

The list of known unknowns is a lot longer, but I think that’s enough to make the problem clear. From popular discussions, you’d think we’re living in an era when anything and everything is routinely quantified and the only problem left is finding the signal in all that noise. For some questions in some (rich) countries, that’s a fair description. For many of the big questions in comparative politics and international relations, though, we’re only just starting to exit the Dark Ages, and the past–and often even the present–are essentially lost to statistical analysis.

Leave a comment

2 Comments

  1. On the other hand, ISTM there are a number of important questions where there’s enough data to engage in reasonable supposition if not give definitive answers — for ex., I like what Thomas Pogge, a philosopher btw not a social scientist, has done in using available data to make arguments about growth & development, etc. See the most recent post on my blog for this.

    Reply
  1. Road-Testing GDELT as a Resource for Monitoring Atrocities | Dart-Throwing Chimp

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: