Geography Students Beat CIA to bin Laden…Or Not

On Twitter today, a couple of people I follow posted links to a blog post claiming that a group of college students had, as part of a biology-class project, pinpointed Osama bin Laden’s whereabouts years before traditional intelligence work finally did. The story is provocatively titled, “This Class of Geography Students Found Bin Laden’s Hideout Long Before the CIA,” and it opens like this:

Two years ago, a class of UCLA undergrads pretty accurately predicted the the location where Osama Bin Laden was hiding out. The students, working under UCLA geography professors Thomas Gillespie and John Agnew, used geographical theories and GIS software to home in on the world’s most wanted fugitive. Science Insider explains: “According to a probabilistic model they created, there was an 80.9% chance that bin Laden was hiding out in Abbottabad, Pakistan, where he was killed last night. And they correctly predicted that he would be in a large town, not a cave.”

I think the story got traction because it fits a narrative we Americans seem to love: “Whiz kids outfox dumb ol’ government!” It’s got tech-savvy youngsters using “outside the box” thinking to beat a bunch of flat-footed bureaucrats at their own game. I don’t know why we like this narrative so much, but we do. In pop culture, the movie WarGames springs to mind, where Matthew Broderick plays a lovable geek who naively hacks into the U.S. military’s supercomputers and nearly starts a nuclear war, then uses his gumption and smarts to teach us all a valuable lesson about the folly of doing so. It also fits with our broader fascination with entrepreneurial nerds and our demonization of Big Government.

Two things bother me about this story, though. First, it’s not true. A correction to the story on the originating site, Science Insider, tells us that the students had determined that bin Laden was probably in the general area of Pakistan where he was finally found, but not with much precision. Contrary to the Good version, they had not pinpointed his whereabouts in the town of Abbottabad. Instead, they had merely assigned a 88.9% likelihood to his being within 300km of his last known location, Tora Bora — a massive circular area that happens to include the town of Abbottabad. When it comes to locating a single individual, it’s hard to see how that level of specificity (or lack thereof) is an improvement over the widely shared sense that bin Laden was probably hiding in Pakistan, possibly in the area near the border with Afghanistan. As intel types might say, their result was not exactly “actionable.”

Second, even if the students had been (more) right, it’s not clear how that result could have helped U.S. intelligence agencies locate bin Laden any sooner. Here’s the problem: the project described in that newspaper article is probably one of scores, maybe hundreds, of amateur and professional efforts over the past decade that analyzed open-source data to try to figure out where bin Laden was hiding. Once we know where bin Laden actually was, it’s easy to comb back through all those projects and find one or a few that got pretty warm, maybe even hot, in that search.

Now, though, try putting youself in the shoes of someone trying to figure out how to use the information from all of those projects before we know where bin Laden is. Whose results do you believe, and why? These projects amount to forecasts, and the normal way to adjudicate among competing forecasts is to compare their accuracy in the past. But when the event of interest is a one-shot deal — the location of a single individual at a specific point in time — there’re no track records to compare. At best, you have a bunch of vague leads that are probably no better than the ones you’re already getting from human and signals intelligence, and maybe even worse. In other words, instead of a clear but overlooked signal, all you’ve really got is even more noise.

(UPDATE: For more on how bin Laden’s compound was actually located by intelligence professionals at the National Geospatial-Intelligence Agency, see this story by Marc Ambinder at NationalJournal.com.)

(UPDATE 2: For details of CIA surveillance of the bin Laden compound from a safe house in Abbottabad, see this story by Greg Miller in The Washington Post.)

(UPDATE 3: Danger Room has this careful deconstruction of the scientific paper that came from the student project which sparked this whole tale. QED.)

Leave a comment

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: