Thursday, December 18, 2008

Stats-crunching finds a new hero

Stats-crunching finds a new hero


According to Nate Silver, you should keep a serious eye on the Cleveland Indians and Detroit Tigers in 2009.

And if you've been paying any attention to politics in the tumultuous year of 2008, you probably already know that Silver is a guy worth listening to.

Silver is a household name to the growing sect of baseball fans known as sabermetricians. He is one of the managing partners of the statistic-bible called Baseball Prospectus.

He developed the PECOTA (Player Empirical Comparison and Optimization Test Algorithm) system of projecting players' future performance, the QERA, a defense-independent pitching statistic, and he writes a well-received Web column urging baseball fans to look beyond conventional statistics that he believes are now far behind the times.

And on Nov. 4, 2008, Silver stunned the political world when his political projection site, FiveThirtyEight.com, nailed Barack Obama's victory over John McCain in the presidential election better than any poll could ever dream of.

FiveThirtyEight.com, named for the total amount of electoral college votes during the presidential election, predicted that Obama would win by 6.1 percentage points (he won by 7.0) and was correct in 49 of the 50 individual states plus the District of Columbia.

Silver says the business of statistical prognostication in baseball and politics aren't as far apart as you might think.

"A lot of what we do at Baseball Prospectus is try and debunk or improve upon conventional wisdom," Silver says. "You see things about clutch hitting or batting average vs. on-base percentage, and we're trying to build a better mousetrap. Discussions in papers or on radio is not as well-informed as to what really goes into winning baseball games.

"And having experienced that for a lot of years, a lot of same conclusions really hold in politics. CNN or Fox or any other networks' discussions are not really that well-informed. Conventional wisdom has people overreacting to randomness in polls. I wanted to create a better-informed discussion."

He did it the same way he calculates baseball stats, through meticulous research of past performance and a few things he learned on the way to Election Day.

"We started out taking average of state polls," Silver says. "But we did one thing differently from other sites. We weighted polls based on which pollsters have been better in the past. All we did at first was take those polls, average the 50 states, and simulate different scenarios.

"But as we got more into it, it got more involved with demographics and started to guess a little more how the polls would go."

Silver has been fascinated by numbers -- and baseball -- for all 30 of his years.

Growing up in Michigan, he fell in love with the Detroit Tigers when they won the 1984 World Series, and he began playing fantasy baseball as a teenager. He graduated with honors in economics from the University of Chicago, and he has been helping revolutionize the statistic-based approach to baseball ever since.

He acknowledges that while many modern baseball minds have embraced new-school work pioneers such as Bill James and the theories that he and his Prospectus colleagues have been working on for decades, many "old-school" baseball minds who rely solely on the reports of scouts in the field have rejected it.

He says it's not quite like that in politics.

"Campaigns are pretty sophisticated," Silver says. "They know how to use data. In the Obama campaign, for example, a quarter of the employees were working on data. It was a quantitatively oriented campaign. And it's not just Democrats. Karl Rove is a big data geek. And that's the one thing winning campaigns share.

"So you don't really have the tension like you've seen in the conflicts between stats and scouts."

But well-run baseball teams have to have a balance, Silver admits.

"You look at the Red Sox or Rays, and they both have scouts and input from stats guys," Silver says. "They're using resources, which is what well-run political campaigns do. The good campaigns understand that to begin with. The poorly run ones don't."

Silver's dead-on projections for the 2008 election have earned him numerous TV appearances, including MSNBC's "Countdown with Keith Olbermann" and "The Rachel Maddow Show," Comedy Central's "The Colbert Report" and CNN's "American Morning," plus an upcoming multi-book deal with a major publisher.

But he's not resting on his laurels right now.

"I'm working on the PECOTA forecasts for next year," Silver says. "We have to get our book to the publisher by Christmas."

In the meantime, Silver says he's happy to be back talking baseball.

"One thing that's nice about baseball is that you don't run into people who are necessarily ideological," Silver says.

"And I guess baseball is a little more fun than politics, too."

No comments: