All posts in category Mass Atrocities

Challenges in Measuring Violent Conflict, Syria Edition

As part of a larger (but, unfortunately, gated) story on how the terrific new Global Data on Events, Language, and Tone (GDELT) might help social scientists forecast violent conflicts, the New Scientist recently posted some graphics using GDELT to chart the ongoing civil war in Syria. Among those graphics was this time-series plot of violent events per day in Syria since the start of 2011:

Syrian Conflict New Scientist

Based on that chart, the author of the story (not the producers of GDELT, mind you) wrote:

As Western leaders ponder intervention, the resulting view suggests that the violence has subsided in recent months, from a peak in the third quarter of 2012.

That inference is almost certainly wrong, and why it’s wrong underscores one of the fundamental challenges in using event data—whether it’s collected and coded by software or humans or some combination thereof—to observe the dynamics of violent conflict.

I say that inference is almost certainly wrong because concurrent data on deaths and refugees suggest that violence in Syria has only intensified in past year. One of the most reputable sources on deaths from the war is the Syria Tracker. A screenshot of their chart of monthly counts of documented killings is shown below. Like GDELT, their data also identify a sharp increase in violence in late 2012. Unlike GDELT, their data indicate that the intensity of the violence has remained very high since then, and that’s true even though the process of documenting killings inevitably lags behind the actual violence.

Syria Tracker monthly death counts

We see a similar pattern in data from the U.N. High Commissioner on Refugees (UNHCR) on people fleeing the fighting in Syria. If anything, the flow of refugees has only increased in 2013, suggesting that the violence in Syria is hardly abating.

UNHCR syria refugee plot

The reason GDELT’s count of violent events has diverged from other measures of the intensity of the violence in Syria in recent months is probably something called “media fatigue.” Data sets of political events generally depend on news sources to spot events of interest, and it turns out that news coverage of large-scale political violence follows a predictable arc. As Deborah Gerner and Phil Schrodt describe in a paper from the late 1990s, press coverage of a sustained and intense conflicts is often high when hostilities first break out but then declines steadily thereafter. That decline can happen because editors and readers get bored, burned out, or distracted. It can also happen because the conflict gets so intense that it becomes, in a sense, too dangerous to cover. In the case of Syria, I suspect all of these things are at work.

My point here isn’t to knock GDELT, which is still recording scores or hundreds of events in Syria every day, automatically, using open-source code, and then distributing those data to the public for free. Instead, I’m just trying to remind would-be users of any data set of political events to infer with caution. Event counts are one useful way to track variation over time in political processes we care about, but they’re only one part of the proverbial elephant, and they are inevitably constrained by the limitations of the sources from which they draw. To get a fuller sense of the beast, we need as often as possible to cross-reference those event data with other sources of information. Each of the sources I’ve cited here has its own blind spots and selection biases, but a comparison of trends from all three—and, importantly, an awareness of the likely sources of those biases—is enough to give me confidence that the civil war in Syria is only continuing to intensify. That says something important about Syria, of course, but it also says something important about the risks of drawing conclusions from event counts alone.

PS. For a great discussion of other sources of bias in the study of political violence, see Stathis Kalyvas’ 2004 essay on “The Urban Bias in Research on Civil Wars” (PDF).

13 Comments
by dartthrowingchimp on May 16, 2013  •  Permalink
Posted in Mass Atrocities, Methods, Uncategorized, Violent Conflict
Tagged , , , , ,

Posted by dartthrowingchimp on May 16, 2013

http://dartthrowingchimp.wordpress.com/2013/05/16/challenges-in-measuring-violent-conflict-syria-edition/

Road-Testing GDELT as a Resource for Monitoring Atrocities

As I said here a few weeks ago, I think the Global Dataset on Events, Location, and Tone (GDELT) is a fantastic new resource that really embodies some of the ways in which technological changes are coming together to open lots of new doors for social-scientific research. GDELT’s promise is obvious: more than 200 million political events from around the world over the past 30 years, all spotted and coded by well-trained software instead of the traditional armies of undergrad RAs, and with daily updates coming online soon. Or, as Adam Elkus’ t-shirt would have it, “200 million observations. Only one boss.”

BUT! Caveat emptor! Like every other data-collection effort ever, GDELT is not alchemy, and it’s important that people planning to use the data, or even just to consume analysis based on it, understand what its limitations are.

I’m starting to get a better feel for those limitations from my own efforts to use GDELT to help observe atrocities around the world, as part of a consulting project I’m doing for the U.S. Holocaust Memorial Museum’s Center for the Prevention of Genocide. The core task of that project is to develop plans for a public early-warning system that would allow us to assess the risk of onsets of atrocities in countries worldwide more accurately and earlier than current practice.

When I heard about GDELT last fall, though, it occurred to me that we could use it (and similar data sets in the pipeline) to support efforts to monitor atrocities as well. The CAMEO coding scheme on which GDELT is based includes a number of event types that correspond to various forms of violent attack and other variables indicating who was doing attacking whom. If we could develop a filter that reliably pulled events of interest to us from the larger stream of records, we could produce something like a near-real time bulletin on recent violence against civilians around the world. Our record would surely have some blind spots—GDELT only tracks a limited number of news sources, and some atrocities just don’t get reported, period—but I thought it would reliably and efficiently alert us to new episodes of violence against civilians and help us identify trends in ongoing ones.

Well, you know what they say about plans and enemies and first contact. After digging into GDELT, I still think we can accomplish those goals, but it’s going to take more human effort than I originally expected. Put bluntly, GDELT is noisier than I had anticipated, and for the time being the only way I can see to sharpen that signal is to keep a human in the loop.

Imagine (fantasize?) for a moment that there’s a perfect record somewhere of all the political interactions GDELT is trying to identify. For kicks, let’s call it the Encyclopedia Eventum (EE). Like any detection system, GDELT can mess up in two basic ways: 1) errors of omission, in which GDELT fails to spot something that’s in the EE; and 2) errors of commission, in which it mistakenly records an event that isn’t in the EE (or, relatedly, is in the EE but in a different place). We might also call these false negatives and false positives, respectively.

At this point, I can’t say anything about how often GDELT is making errors of omission, because I don’t have that Encyclopedia Eventum handy. A more realistic strategy for assessing the rate of errors of omission would involve comparing a subset of GDELT to another event data set that’s known to be a fairly reliable measure for some time and place of something GDELT is meant to track—say, protest and coercion in Europe—and see how well they match up, but that’s not a trivial task, and I haven’t tried it yet.

Instead, the noise I’m seeing is on the other side of that coin: the errors of commission, or false positives. Here’s what I mean:

To start developing my atrocities-monitoring filter, I downloaded the reduced and compressed version of GDELT recently posted on the Penn State Event Data Project page and pulled the tab-delimited text files for a couple of recent years. I’ve worked with event data before, so I’m familiar with basic issues in their analysis, but every data set has its own idiosyncrasies. After trading emails with a few CAMEO pros and reading Jay Yonamine’s excellent primer on event aggregation strategies, I started tinkering with a function in R that would extract the subset of events that appeared to involve lethal force against civilians. That function would involve rules to select on three features: event type, source (the doer), and target.

  • Event Type. For observing atrocities, type 20 (“Engage in Unconventional Mass Violence”) was an obvious choice. Based on advice from those CAMEO pros, I also focused on 18 (“Assault”) and 19 (“Fight”) but was expecting that I would need to be more restrictive about the subtypes, sources, and targets in those categories to avoid errors of commission.
  • Source. I’m trying to track violence by state and non-state agents, so I focused on GOV (government), MIL (Military), COP (police), and intelligence agencies (SPY) for the former and REB (militarized opposition groups) and SEP (separatist groups) for the latter. The big question mark was how to handle records with just a country code (e.g., “SYR” for Syria) and no indication of the source’s type. My CAMEO consultants told me these would usually refer in some way to the state, so I should at least consider including them.
  • Target. To identify violence against civilians, I figured I would get the most mileage out of the OPP (non-violent political opposition), CVL (“civilians,” people in general), and REF (refugees) codes, but I wanted to see if the codes for more specific non-state actors (e.g., LAB for labor, EDU for schools or students, HLH for health care) would also help flag some events of interest.

After tinkering with the data a bit, I decided to write to separate functions, one for events with state perpetrators and another for events with non-state perpetrators. If you’re into that sort of thing, you can see the state-perpetrator version of that filtering function on Github, here.

When I ran the more than 9 million records in the “2011.reduced.txt” file through that function, I got back 2,958 events. So far, so good. As soon as I started poking around in the results, though, I saw a lot of records that looked . The current release of GDELT doesn’t include text from or links to the source material, so it’s hard to say for sure what real-world event any one record describes. Still, some of the perpetrator-and-target combos looked odd to me, and web searches for relevant stories either came up empty or reinforced my suspicions that the records were probably errors of commission. Here are a few examples, showing the date, event type, source, and target:

  • 1/8/2011 193 USAGOV USAMED. Type 193 is “Fight with small arms and light weapons,” but I don’t think anyone from the U.S. government actually got in a shootout or knife fight with American journalists that day. In fact, that event-source-target combination popped up a lot in my subset.
  • 1/9/2011 202 USAMIL VNMCVL. Taken on its face, this record says that U.S. military forces killed Vietnamese civilians on January 9, 2011. My hunch is that the story on which this record is based was actually talking about something from the Vietnam War.
  • 4/11/2011 202 RUSSPY POLCVL. This record seems to suggest that Russian intelligence agents “engaged in mass killings” of Polish civilians in central Siberia two years ago. I suspect the story behind this record was actually talking about the Kaytn Massacre and associated mass deportations that occurred in April 1940.

That’s not to say that all the records looked wacky. Interleaved with these suspicious cases were records representing exactly the kinds of events I was trying to find. For example, my filter also turned up a 202 GOV SYRCVL for June 10, 2011, a day on which one headline blared “Dozens Killed During Syrian Protests.”

Still, it’s immediately clear to me that GDELT’s parsing process is not quite at the stage where we can peruse the codebook like a menu, identify the morsels we’d like to consume, phone our order in, and expect to have exactly the meal we imagined waiting for us when we go to pick it up. There’s lots of valuable information in there, but there’s plenty of chaff, too, and for the time being it’s on us as researchers to take time to try to sort the two out. This sorting will get easier to do if and when the posted version adds information about the source article and relevant text, but “easier” in this case will still require human beings to review the results and do the cross-referencing.

Over time, researchers who work on specific topics—like atrocities, or interstate war, or protest activity in specific countries—will probably be able to develop supplemental coding rules and tweak their filters to automate some of what they learn. I’m also optimistic that the public release of GDELT will accelerate improvements the software and dictionaries it uses, expanding its reach while shrinking the error rates. In the meantime, researchers are advised to stick to the same practices they’ve always used (or should have, anyway): take time to get to know your data; parse it carefully; and, when there’s no single parsing that’s obviously superior, check the sensitivity of your results to different permutations.

PS. If you have any suggestions on how to improve the code I’m using to spot potential atrocities or otherwise improve the monitoring process I’ve described, please let me know. That’s an ongoing project, and even marginal improvements in the fidelity of the filter would be a big help.

PPS. For more on these issues and the wider future of automated event coding, see this ensuing post from Phil Schrodt on his blog.

5 Comments
by dartthrowingchimp on May 2, 2013  •  Permalink
Posted in Mass Atrocities, Methods, Statistics
Tagged , , ,

Posted by dartthrowingchimp on May 2, 2013

http://dartthrowingchimp.wordpress.com/2013/05/02/road-testing-gdelt-as-a-resource-for-monitoring-atrocities/

Kenya: An Ounce of Prevention or a Pound of Overreaction?

On March 4, Kenya held general elections, and nearly no one was killed. That might not sound like a big deal, but lots of smart people had been warning for months that these elections put Kenya at high risk of mass atrocities.

Assuming Kenya stays the course and completes the current election cycle without large-scale violence, the big question for people concerned about atrocities prevention is this: Did all the scrutiny and alarm help to prevent violence that would otherwise have occurred, or did we collectively overreact to the surprise of early 2008 and cry “Wolf!” when none was near?

Line to vote at the Old Kibera Primary School on March 4, 2013 (Georgina Goodwin, AFP/Getty Images)

Line to vote at the Old Kibera Primary School on March 4, 2013 (Georgina Goodwin, AFP/Getty Images)

I emailed this question to Ken Opalo, a Stanford Ph.D. candidate who’s from Kenya and was there to analyze and vote in the elections, and he offered a favorable assessment of the many preventive efforts. “I think the peace crusade actually helped prevent violence by constantly reminding us of the cost of violence,” he said. Ken also credited the Kenyan media for choosing not to air inflammatory political statements and the government for blocking the dissemination of hate speech via short message service (SMS), an important channel of communication . Last but not least, Ken argued that the dynamics of the presidential campaign also played a role. “It also helps,” he wrote, “that one of the most volatile regions in the country—the central Rift Valley—this time round found peace in the political union between [eventual winner Uhuru] Kenyatta and his deputy William Ruto (bringing together Kikuyus and Kalenjins).”

Kenyan columnist Charles Onyago-Obbo also believes that reactions to the  helped to avert violence that might have been. In a column entitled “Why Kenyans didn’t run berserk,” he acknowledges that peace campaigns by social groups and the media may have helped at the margins, but he sees the biggest effects coming from sticks and carrots deployed by the Kenyan government. Like Opalo, he credits authorities’ crackdown on hate speech, but he also believes that visible investments in major infrastructural projects in some key regions also had a significant effect.

If we believe that Kenyans became more good-hearted, then to prevent future violence, it would be necessary to preach more peace, hold peace concerts, and keep warning about the dangers of a repeat of 2008.

If we believe that people respond to incentives and symbols of progress, then the correct policy is to build more roads, fix more airports, complete Konza City and start a second one, keep working at political reform, and walk around with a big stick to crack the skulls of hate entrepreneurs.

I am a structuralist; I am in the last camp.

International actors are also claiming some credit. Luis Moreno-Ocampo, the former chief prosecutor for the International Criminal Court, told the Associated Press that the prosecutions he pursued after 2008 “were game-changers that helped prevent a repeat of the deadly rampages following this month’s vote.” Moreno-Ocampo saw his office as an instrument of deterrence, and in this case, he believes it worked.

I emailed Daniel Solomon, a Georgetown University senior who is both a student and advocate of atrocities prevention, to ask him how influential the ICC indictments had been. He agreed with Ocampo that they had some effect, but he described that effect as indirect:

I don’t think there was a substantial risk of violence by Kenyatta’s close affiliates, or by [second-place finisher Raila] Odinga’s: if you look at the financial and political incentives for national-level officials, many are linked to international investments in Kenya’s economy, or to aid flows by Western donor states. This is probably less the case for [members of parliament] who weren’t as internationally prominent, and those are often the officials with the most direct links to paramilitary forces and civilian militias.

As a result, I think we can differentiate between a couple of dynamics, each of which had a unique function in the context of violence prevention: the intergovernmental preparation, which was both public at a national level (high-level diplomatic statements, threats of consequences for violence) and “behind-the-scenes”; and the non-governmental preparation, which was both public at the national level (statements about the ICC, human rights reporting) and “behind-the-scenes” at a local level (it’s hard to assess whether this was marginal, or structurally important). In one sense, it’s hard to draw a hard line between the two, but I’m not sure the non-governmental commentary would have been influential in changing those local MP incentives without an active intergovernmental process behind the scenes.

tl; dr: We were probably crying wolf where the ICC indictees were concerned (notice how that was always included in press coverage, as if that implies something about anticipated behavior), but I think that process—call it discursive, but there was also tangible diplomacy to back that up—helped diffuse the incentives for violence prevention at more local levels of governance/mobilization.

Personally, I also see the Kenyan elections as a success for atrocities prevention. Large-scale violence was a plausible threat; many efforts were undertaken to prevent that violence; and then it didn’t happen. We can’t say with confident exactly which effort contributed how much, but the risk was real, and the interventions that were undertaken were relatively cheap.

Still, it’s not clear how generalizable this success is. In terms of atrocities risk and prevention, Kenya was exceptional in a couple of important ways. First, this was election-related violence, not insurgency or civil war. That meant that the risk was tied to a specific political process with clear milestones and outcomes and was not part of a deeper syndrome of insecurity and mass violence. Second, the Kenyan government was a willing partner in atrocities prevention instead of a perpetrator.

Those two features make Kenya in 2013 very different from places like Syria or Sudan, where state security forces and their fellow travelers are doing the killing and the governments involved reject outside interference. Future attempts to prevent election-related and other “communal” violence might look to this case to try to understand why the Kenyan government was a willing partner and which components seem to have been most effective, but I don’t think there are big lessons from Kenya that can be transferred to more typical cases of concern. To see what I mean, just think about how effective an ICC indictment has been at preventing atrocities in Sudan, or how effective hate-speech monitoring would be at stopping violence in Syria.

Finally, it’s important to recognize that, while relatively cheap, these efforts were not cost free. In a post on the New York Times‘ Latitude blog, Journalist Michaela Wrong argues that self-censorship by the Kenyan media around this month’s elections diminished the country’s democracy.

“Last time,” the media “were part of the problem,” a Kenyan broadcaster told me. “They were corrupted; they were irresponsible. So this time there was a feeling that we had to keep everyone calm, at the expense, if necessary, of our liberties.”

But self-censorship comes at a price: political impartiality. The decision not to inflame ethnic passions meant that media coverage shifted in favor of whoever took an early lead, in this case Uhuru Kenyatta.

That’s an important reminder that policy interventions often entail trade-offs across values we might think of as complementary instead of competing. Democratization and atrocities prevention are both things many of us would espouse, but what’s good for one won’t always be good for the other.

5 Comments
by dartthrowingchimp on March 18, 2013  •  Permalink
Posted in Mass Atrocities, Violent Conflict
Tagged , , , ,

Posted by dartthrowingchimp on March 18, 2013

http://dartthrowingchimp.wordpress.com/2013/03/18/kenya-an-ounce-of-prevention-or-a-pound-of-overreaction/

A Cautionary Note on Increased Aid to Syrian Rebels

According to today’s Washington Post, the U.S. government is starting to supply food and medicine directly to selected Syrian rebel groups. Meanwhile, “Britain and other nations working in concert with the United States are expected to go further to help the rebel Free Syrian Army by providing battlefield equipment such as armored vehicles, night-vision devices or body armor.”

The point of all this assistance, of course, is to hasten the fall of Syrian President Bashir al-Assad. According to newly minted Secretary of State John Kerry, Assad is “out of time and must be out of power.”

220px-Ford_assembly_line_-_1913

Best I can tell, the logic behind this stepped-up support for the Syrian rebels Western governments “like” follows the logic of an assembly line. To increase desired outputs, increase relevant inputs.

But civil wars aren’t like factories. They’re more like ecosystems, and if there’s one thing we’ve learned from our attempts to manage ecosystems, it’s that they often have unintended consequences. Consider this 2009 story from the New York Times:

With its craggy green cliffs and mist-laden skies, Macquarie Island — halfway between Australia and Antarctica — looks like a nature lover’s Mecca. But the island has recently become a sobering illustration of what can happen when efforts to eliminate an invasive species end up causing unforeseen collateral damage.

In 1985, Australian scientists kicked off an ambitious plan: to kill off non-native cats that had been prowling the island’s slopes since the early 19th century. The program began out of apparent necessity — the cats were preying on native burrowing birds. Twenty-four years later, a team of scientists from the Australian Antarctic Division and the University of Tasmania reports that the cat removal unexpectedly wreaked havoc on the island ecosystem.

With the cats gone, the island’s rabbits (also non-native) began to breed out of control, ravaging native plants and sending ripple effects throughout the ecosystem. The findings were published in the Journal of Applied Ecology online in January.

“Our findings show that it’s important for scientists to study the whole ecosystem before doing eradication programs,” said Arko Lucieer, a University of Tasmania remote-sensing expert and a co-author of the paper. “There haven’t been a lot of programs that take the entire system into account. You need to go into scenario mode: ‘If we kill this animal, what other consequences are there going to be?’”

I don’t mean to suggest a moral equivalence between the human beings fighting and being murdered in Syria and the rabbits and cats and birds on Macquarie Island. I do mean to suggest that attempts to manipulate systems like these almost always underestimate the complexity of the problem. What scientist Barry Rice said to the New York Times for that 2009 article on the difficulty of managing invasive species applies just as well to attempts by outside powers to manufacture desired outcomes in civil wars:

When you’re doing a removal effort, you don’t know exactly what the outcome will be. You can’t just go in and make a single surgical strike. Every kind of management you do is going to cause some damage.

I hope Syria gets to a better place soon. Like Dan Trombly and Ahsan Butt, however, I am not confident that increased support for selected rebel factions will help that happen, and I am worried about the unintended consequences it will bring.

3 Comments
by dartthrowingchimp on February 28, 2013  •  Permalink
Posted in Complexity Politics, Mass Atrocities, Revolutions
Tagged

Posted by dartthrowingchimp on February 28, 2013

http://dartthrowingchimp.wordpress.com/2013/02/28/a-cautionary-note-on-increased-aid-to-syrian-rebels/

Do Elections Trigger Mass Atrocities?

Kenya plans to hold general elections in early March this year, and many observers fear those contests will spur a reprisal of the mass violence that swept parts of that country after balloting in December 2007.  The Sentinel Project for Genocide Prevention says Kenya is at “high risk“ of genocide in 2013, and a recent contingency-planning memo from Joel Barkan the Council on Foreign Relations asserts that “there will almost certainly be further incidents of violence in the run-up to the 2013 elections.” As a recent Africa Initiative backgrounder points out, this violence has roots that stretch much deeper than the 2007 elections, but the fear that mass violence will flare again around this year’s balloting seems well founded.

All of which got me wondering: is this a generic problem? We know that election-related violence is a real and multifaceted thing. We also have works by Jack Snyder and Amy Chua, among others, arguing that democratization actually makes some countries more susceptible to ethnic and nationalist conflict rather than less, as democracy promoters often claim. What I’m wondering, though—as someone who has long studied democratization and is currently working on tools to forecast genocide and other forms of mass killing—is whether or not elections substantially increase the risk of mass atrocities in particular, where “mass atrocities” means the deliberate killing of large numbers of unarmed civilians for apparently political ends.

Best I can tell, the short answer is no. After applying a few different statistical-modeling strategies to a few measures of atrocities, I see little evidence that elections commonly trigger the onset or intensification of this type of political violence. The absence of evidence isn’t the same thing as evidence of absence, but these results convince me that national elections aren’t a major risk factor for mass killing.

If you’re interested in the technical details, here’s what I did and what I found:

My first cut at the problem looked for a connection between national elections and the onset of state-sponsored mass killings, defined as “a period of sustained violence” in which “ the actions of state agents result in the intentional death of at least 1,000 noncombatants from a discrete group.” That latter definition comes from work Ben Valentino and I did for my old research program, the Political Instability Task Force, and it restricts the analysis to episodes of large-scale killing by states or other groups acting at their behest. Defined as such, mass killings are akin to genocide in their scale, and there have only been about 110 of them since 1945.

So, do national elections help trigger this type of mass killing? To try to answer this question, I thought of elections as a kind of experimental “treatment” that some country-years get and others don’t. I used the National Elections Across Democracy and Autocracy (NELDA) data set to identify country-years since 1945 with national elections for chief executive or legislature or both, regardless of how competitive those elections were. I then used the MatchIt package in R to set up a comparison of country-years with and without elections within 107 groups that matched exactly on several other variables identified by prior research as risk factors for mass-killing onset: autocracy vs. democracy, exclusionary elite ideology (yes/no), salient elite ethnicity (yes/no), ongoing armed conflict (yes/no), any mass killing since 1945 (yes/no), and Cold War vs. post-Cold War period. Finally, I used conditional logistic regression to estimate the difference in risk between election and non-election years within those groups.

The results? In my data, mass-killing episodes were 80% as likely to begin in election years as non-election years, other things being equal. The 95% confidence interval for this association was wide (45% to 145%), but the result suggests that, if anything, countries are actually somewhat less prone to suffer onsets of mass killing in election years as non-election years.

I wondered if the risk might differ by regime type, so I reran the analysis on the subset of cases that were plausibly democratic. The estimate was effectively unchanged (80%, CI of 35% to 185%). Then I thought it might be a post-Cold War thing and reran the analysis using only country-years from 1991 forward. The estimate moved, but in the opposite of the anticipated direction. Now it was down to 60%, with a CI of 17% to 215%.

These estimates got me worried that something had gone wacky in my data, so I reran the matching and conditional logistic regression using coup attempts (successful or failed) instead of elections as the “treatment” of interest. Several theorists have identified threats to incumbents’ power as a cause of mass atrocities, and coups are a visible and discrete manifestation of such threats. My analysis strongly confirmed this view, indicating that mass-killing episodes were nearly five times as likely to start in years with coup attempts as years without, other things being equal. More important for present purposes, this result increased my confidence in the reliability of my earlier finding on elections, as did the similar estimates I got from models with country fixed effects, country-specific intercepts (a.k.a. random effects), and interaction terms that allowed the effects of elections to vary across regime types and historical eras.

Then I wondered if this negative finding wasn’t an artifact of the measure I was using for mass atrocities. The 1,000-death threshold for “mass killing” is quite high, and the restriction to killings by states or their agents ignores situations of grave concern in which rebel groups or other non-state actors are the ones doing the murdering. Maybe the danger of election years would be clearer if I looked at atrocities on a smaller scale and ones perpetrated by non-state actors.

To do this, I took the UCDP One-Sided Violence Dataset v1.4 and wrote an R script that aggregated its values for specific conflicts into annual death counts by country and perpetrator (government or non-government). Then I used R’s ‘pscl’ package to estimate zero-inflated negative binomial regression (ZINB) models that treat the death counts as the observable results of a two-stage process: one that determines whether or not a country has any one-sided killing in a particular year, and then another that determines how many deaths occur, conditional on there being any. In addition to my indicator for election years, these models included all the risk factors used in the earlier matching exercise, plus population size and the logged counts of deaths from one-sided violence by government and non-government actors (separately) in the previous year. All of these variables were included in the logistic regression “hurdle” model; only elections, population size, and the lagged death counts were included in the conditional count models.

To my surprise once again, the results suggested that, if anything, atrocities the risk of mass atrocities is actually lower in years with national elections. In the model of government-perpetrated violence, the coefficient for the election indicator in the hurdle model was barely distinguishable from zero (0.04), and the association in the count portion was modestly negative (-0.20, s.e. of 0.20). In the model of violence perpetrated by other groups, the effect in the hurdle portion was modestly negative (-0.25, s.e. of 0.20), and the effect in the count portion was decidedly negative (-0.82, s.e. of 0.19). When I reran the models with separate indicators for executive and legislative elections, the results bounced around a little bit, but the basic patterns remained unchanged. None of the models showed a substantial, positive association between either type of election and the occurrence or scale of one-sided violence against civilians.

In light of the weakness of the observed effects, the noisiness of the measures employed, and my prior beliefs about the effects of elections on risks of mass killing—shaped in part by the Kenyan case I discussed at the start of this post—I’m not quite ready to assert that election years actually reduce the risk of mass atrocities. What I am more comfortable doing, however, is ignoring elections in statistical models meant to forecast mass atrocities across large numbers of countries.

If you’re interested in replicating or tweaking this analysis, please email me at ulfelder@gmail.com, and I’ll be happy to send you the data and R scripts (one to get country-year summaries of the UCDP data, another to run the matching and modeling) I used to do it. [UPDATE: I've put the scripts and data in a publicly accessible folder on Google Drive. If you try that link and it doesn't work, please let me know.] Ideally, I would cut out the middleman by putting them in a Github repository, but I haven’t quite figured out how to do that yet. If you’re in the DC area and interested in getting paid to walk me me through that process, please let me know.

10 Comments
by dartthrowingchimp on January 14, 2013  •  Permalink
Posted in Democratization, Mass Atrocities
Tagged , , , , , ,

Posted by dartthrowingchimp on January 14, 2013

http://dartthrowingchimp.wordpress.com/2013/01/14/do-elections-trigger-mass-atrocities/

Building a Public Early-Warning System for Genocide and Mass Atrocities

Can we see genocides and other mass atrocities coming? If so, how, and how far in advance? And would public dissemination of those forecasts help policy-makers, advocates, and affected societies prevent those atrocities from occurring?

In October 2011, the U.S. Holocaust Memorial Museum (USHMM) convened a group of advocates and academics for a one-day seminar to ruminate on these questions. These are big and difficult problems, and the event really had a more practical goal at its heart: to help the Museum and other civil-society groups assess the potential for, and value of, a new public early-warning system focused on genocide and other mass atrocities.

Based on that conversation and the recommendations of USHMM Fellow and Dartmouth professor Ben Valentino, the Museum decided that the need and opportunity were sufficient to start considering what such a system might look like and how to build it. In March 2012, the Museum hired me for an eight-month consulting project, to finish in October, that’s meant to push this process forward.

My project has two main parts. First and most important, I’ve been asked to write a prospectus detailing the elements and funding this program would require. Second, I’ve been asked to build a statistical tool that could produce one set of forecasts for this program, if it gets built. Under Ben’s proposal, a second set of forecasts would come from some form of expert survey, and the two could be compared and combined to useful effect.

As I get deeper into the project, I expect to blog occasionally about what I’m working on and where I could use some help. I’ve already had very helpful exchanges with numerous people engaged in related projects, including former Political Instability Task Force colleagues Ted Gurr and Barbara Harff, who produces her own global genocide risk list each year, and Sentinel Project founder Christopher Tuckwood. I’m also slated to present results from a preliminary version of my statistical analysis at NYU’s Northeast Methods Program (NEMP) in early May, and my work will surely benefit from the constructive criticism that esteemed audience can provide.

In the meantime, I wanted to spread the word about the Museum’s interest in this endeavor and invite your reactions and suggestions. If you know of any relevant research or advocacy projects or might be interested in supporting this work in some fashion, please post a comment or drop me a line at ulfelder <at> gmail <dot> com.

19 Comments
by dartthrowingchimp on April 7, 2012  •  Permalink
Posted in Forecasting, Mass Atrocities, Statistics
Tagged , , , , , ,

Posted by dartthrowingchimp on April 7, 2012

http://dartthrowingchimp.wordpress.com/2012/04/07/building-a-public-early-warning-system-for-genocide-and-mass-atrocities/

Follow

Get every new post delivered to your Inbox.

Join 3,481 other followers

%d bloggers like this: