Over the past year, I’ve watched a few people I know in digital life sink a fair amount of time into statistical modeling projects that other people might see as “just for fun,” if not downright frivolous. Last April, for example, public-health grad student Brett Keller delivered an epic blog post that used event history models to explore why some competitors survive longer than others in the fictional Hunger Games. More recently, sociology Ph.D. student Alex Hanna has been using the same event history techniques to predict who’ll get booted each week from the reality TV show RuPaul’s Drag Race (see here and here so far). And then there’s Against the Spread, a nascent pro-football forecasting project from sociology Ph.D. candidate Trey Causey, whose dissertation uses natural language processing and agent-based modeling to examine information ecology in authoritarian regimes.
I happen to think these kinds of projects are a great idea, if you can find the time to do them–and if you’re reading this blog post, you probably can. Based on personal experience, I’m a big believer in learning by doing. Concepts don’t stick in my brain when I only read about them; I’ve got to see the concepts in action and attach them to familiar contexts and examples to really see what’s going on. Blog posts like Brett’s and Alex’s are a terrific way to teach yourself new methods by applying them to toy problems where the data sets are small, the domain is familiar and interesting, and the costs of being wrong are negligible.
A bigger project like Trey’s requires you to solve a lot of complex procedural and methodological problems, but all the skills you develop along the way transfer to other domains. If you can build and run a decent forecasting system from scratch for something as complex as pro football, you can do the same for “seriouser” problems, too. I think that demonstrated skill on fun tasks says as much about someone’s ability to execute complex research in the real world as any job talk or publication in a peer-reviewed journal. Done well, these hobby projects can even evolve into rewarding enterprises of their own. Just ask Nate Silver, who kickstarted his now-prodigious career as a statistical forecaster with PECOTA, a baseball forecasting system that he ginned up for fun while working for pay as a consultant.
I suspect that a lot of people in the private sector already get this. Academia, not so much, but then they’re the ones who wind up poorer for it.
Dart-Throwing Chimp 
Alex
/ March 20, 2013Thanks for the mention, Jay! Echoing your sentiments, doing the RPDR project has been very fun and a good way to learn methods that I’m generally unfamiliar with. It also keeps me locked into updating BadHessian.org at least once a week, which is probably good blogging practice.
Oral Hazard
/ March 20, 2013Do you think linking to this post would help when my wife grouses about my allegedly endless and frivolous engagement with fantasy baseball? I’m thinking no, but I appreciate the effort and it confirms my own thinking. 🙂
dartthrowingchimp
/ March 20, 2013Sorry, bro, but I think you only get experience points from fantasy baseball if you use a multilevel model to help select your team, or build a scraper to deliver custom updates, or do something equally hardcore.
Oral Hazard
/ March 20, 2013But I do use a multilevel model to help select my team, courtesy of the folks at Baseball Forecaster. Oh, you mean it has to be MY multilevel model…
In all seriousness, though, I think there’s a lot to be said for playful hobbying scaled to the more modest skill level of the participant, where the hobbying stretches the mind and enhances transferable skills. I don’t think its a non sequitur to say I truly believe that being humbled by 15 fantasy baseball seasons has made me a better lawyer.
Can I convince my wife of that? No chance. None. Really enjoy the blog.
Trey Causey
/ March 20, 2013Thanks for the mention, Jay. Good points as always. I think you’re right on about the private sector and not academia recognizing the value of intellectually challenging projects that may not fit in the nominal category of “serious work.” The idea that one must always be narrowly focused on one’s academic specialization is, in my view, based on a straw man conception of how scientists actually operate. It’s unfortunate, because we know this, but still feel the need to hold ourselves or others to this standard.
dartthrowingchimp
/ March 20, 2013For another example that popped up today, check out this application of survival analysis (I’m seeing a theme here) to the NCAA basketball tournament: http://harvardsportsanalysis.wordpress.com/2013/03/19/survival-of-the-fittest-predicting-the-2013-ncaa-tournament/
Brett Keller
/ March 25, 2013I’ve also found that this sort of thing — fun analysis that doesn’t really ‘matter’ — goes much, much faster than you’d expect. The Hunger Games piece, epic as it may be (thanks!), was written almost entirely in one sitting and was posted within 24 hrs of having the idea, whereas I have ‘serious’ blog post that have been saved as drafts for a year.
There’s also a lot of value in fun applications of not-normally-fun projects, like Hangman for Stata (http://www.youtube.com/watch?v=wBmCBjLora0) that I wish was incorporated into more stats/tech education.