AP reports today that the World Anti-Doping Agency (WADA) is conducting an “extraordinary” audit of Jamaica’s drug-testing agency after allegations surfaced that the Jamaican organization had failed to do its job for most of the six months leading to the London Games.
“There was a period of — and forgive me if I don’t have the number of months right — but maybe five to six months during the beginning part of 2012 where there was no effective operation,” WADA Director General David Howman said in an interview. “No testing. There might have been one or two, but there was no testing. So we were worried about it, obviously.”
As fan of track and field and of cycling, I read AP’s story and got a little sadder. At this point, you can’t see stellar performances from guys like Usain Bolt and wonder if banned drugs are what gave those superstars their crucial edge, and failures like this one don’t inspire much confidence.
As a professional observer of democratization, though, I read the AP story and was reminded of the challenges of international election monitoring. Both anti-doping and international election observation efforts involve under-resourced and overly-politicized watchdogs deploying occasional and imperfect tests to try to catch determined cheaters whose careers hang in the balance. Because the stakes are so high, the screening systems we devise are tuned to favor the cheaters. We tolerate errors of omission, or false negatives, to avoid accidentally ruining the reputations of people who aren’t doping or rigging elections, but in so doing, we tolerate a higher rate of cheating than I think most of us realize.
In sport, the bias in the system is encapsulated in the defensive deployment of the phrase “never failed a drug test” by athletes who later admit they cheated. The AP story on Jamaica’s breakdown applies that phrase to world’s fastest man Usain Bolt, and “never a failed test” was a favorite weapon of cyclist Lance Armstrong’s right up until he finally confessed to years of doping.
As statistician Kaiser Fung argues, the fact that an athlete has passed lots of drug tests doesn’t tell us a whole lot when the tests are deliberately skewed to minimize the chance of falsely accusing a “clean” athlete. When we set the threshold for a positive test very high, we create a system in which most cheaters will test negative most of the time. Under these conditions, even a large number of passed tests isn’t especially informative, and the circumstantial evidence—the stories from Lance Armstrong’s trainers and teammates, or the peculiar collapse of Jamaican drug-testing during a critical training period ahead of the London games—should be considered as well.
The election-monitoring equivalent of “never failed a drug test” is the phrase “largely free and fair.” International election observation missions typically deploy staffs of a couple dozen people that are assembled in an ad hoc fashion and have to cover a wide range of issues across whole countries. The missions often expand greatly around election day, but most polling sites still go unobserved, and technical prowess is not necessarily the primary consideration in the selection (or self-selection) of those short-term observers. The quality of the resulting arrangements varies widely, but even in the best of cases, these missions leave plenty of room for determined cheaters to fix the process in their favor.
If the main goal of these missions were to cast doubt on suspicious elections, this piecemeal approach would probably work fine. Even these shoestring missions often catch whiffs of foul play and say so in their reports. For better or for worse, though, these missions also serve political and diplomatic functions, and those other concerns often compel them to soft-pedal their criticisms. Observers want to catch cheats, but they also want to avoid becoming the catalysts of a political crisis and don’t want to discourage governments from participating in the international inspections regime. So the system bends to minimize the risk of false accusations, and we end up with a steady stream of “mostly free and fair” topline judgments that agents of electoral fraud and abuse can then repeat like a mantra to defeat or deflate their domestic political opponents.
Unfortunately, there’s no easy fix in either case. As Kaiser Fung also points out, these trade-offs are unavoidable when trying to detect hard-to-observe phenomena. As long as the tests are imperfect, any reduction in the rate of one kind of error will increase the rate of the other kind. You can slide the threshold up and down, but you can’t wish the errors away.
In sport, we have to decide if we care enough about doping to risk damaging the careers of more “innocent” athletes in pursuit of the (probably many) cheaters who are getting away with it under the current system. In election observation, we have to wonder if the international missions’ declarations of “free and fair” have become so devalued that they don’t serve their intended purpose, and if so, to ask if we’re willing to see more governments disengage from the regime in exchange for a sharper signal. If these choices were easy, we wouldn’t still be talking about them.