Will Political Science Get It Wrong This Year?

All the political science literature shows that the state of the economy is the most important predictor of election outcomes. An incumbent will lose in a bad economy. But Obama seems to be doing OK. In fact, there is even some speculation that November could be a landslide win for him. 

If Obama wins in a landslide in November, will we throw 100's of .pdf files of APSA articles in the trash? Or will we have to add some qualifications to the literature? Possible qualifications: The state of the economy does matter, but the economy isn't really that bad right now. The state of the economy matters, except when the challenger is incredibly unpopular and his VP nominee is hiding under his bed.

The political science literature also poo-poos the importance of presidential debates in turning an election. The media doesn't seem to have gotten that memo. Tomorrow's debate is widely seen as critical to breathing life into Romney's campaign. Will it matter? 

UPDATE: Jennifer at APSA sent me a link to a Wash Post article that summarizes predictions from political scientists. Using existent models, most find that Obama will win, though not all. And those who say he'll win predict that it will be a squeeker. Compare those findings with posts that Nate Silver is writing. Right now, Silver predicts that there is an 85% chance that Obama will win. Who's right? 

16 thoughts on “Will Political Science Get It Wrong This Year?

  1. I think the economy is “improving” at an average rate, but from a really low base. If the movement is in the right direction for long enough, it doesn’t matter that the current point is relatively crappy.
    It looks like a lot of these projections were made a long time ago. The one date I saw was “299 days before the election,” which is about last January. Nate Silver’s predictions are updated daily based on the employment reports and GDP data as it happens — not limited by the publication schedule deadlines of “PS: Political Science and Politics.”

    Like

  2. I think the primary* made Romney run too far to the right both to avoid looking like he has no set of core values and to shift back to a set of positions that could win the general.
    * And by primary, I mean the mostly elderly voters who go to these things. My main political views these days all seem to be center around stopping the baby boom from dodging payment. The current movement is my state is to replace property taxes with income taxes and thus stick me with my portion of the school taxes plus that of my retired neighbors living in identical houses but who don’t even need to pay a mortgage. Anyway, I sincerely hope that you can’t win a national election on a platform that reduces to “Nobody over 65 should pay for health care but the Constitution will die if there is any federal role in paying for healthcare for those under 65.”

    Like

  3. Has Nate Silver altered his analysis to provide parameter-determined predictive models? I thought his analysis was a meta-analysis of polls (weighting for reliability, demographics, . . . and combining them).
    The political science models in the NY Times article seem to be parameter driven.
    If I’m right, the models are completely different, and comparing the predictions is like comparing apples and oranges, and, even more they’re different experiments in prediction, with Silver’s model basically testing the quality of polling while the other models try to explain underlying relationships between real world data and election outcomes, to see if they can understand why populations make the decisions they do. Silver’s models are useful for improving polling and for campaigns in trying to use polling to determine whether they’re manipulations are having real effects that will modulate outcomes.
    The wikipedia article says Silver has done predictive models, and argues that measurable economic variable might explain 30% or so of the variance in election outcomes. But I don’t think the numbers on the graphs at the Silver’s NY times site are based on those models. Silver argues that there aren’t enough elections to make good statistical models (and I agree).

    Like

  4. Coupla thoughts:
    Models based on previous elections seem like they’re going to be based on a pretty low n, whereas polls and polls-of-polls/poll-averages will have a much larger n. That’s a good reason to go with the 538 approach.
    Second, aren’t polls showing people saying that while the overall economic situation might be bad, their own situation is getting better? That would be a pro-Obama indicator.
    Third, the Obama campaign is working really hard and doing it in smart ways. Kay at Balloon Juice (for example) has been describing what O’s folks have been doing in Ohio over the course of the last year or more. It’s not the stuff that’s going to make a NYT story (which says more (and not good) about the NYT than about the campaign), but it is the kind of stuff that will win an election. More than one, really, with the down-ballot work.
    Third-and-a-half, if I remember the models, it seems like one of the premises is often “other things being equal…” The job of the campaign of course being to ensure that the other things aren’t equal.

    Like

  5. As an avid Nate Silver reader, I feel competent to say that 538 is doing a bunch of different things. If you look at his charts, he has a “Forecast” and a “Now-Cast.”
    The “Now-Cast” is based solely on polls, and determines “if the election were held today,” how would it come out.
    The “Forecast” is a prediction to Election Day, and takes into account more things than just polling. So, on one day a week ago the polling improved for Obama, but a really bad job number came in, so the “Now-Cast” went up while the “Forecast” went down. The Forecast also considers reversion to the mean, convention bumps subsiding, and the possibility that Obama will state during the debates that he intends to impose Sharia law if re-elected.
    The two numbers slowly converge as the non-poll projections are filtered out, so by election day there is only the polls left

    Like

  6. Really, what most of these models do is make me question the analytical skills of the political science community. Most of them seem to be based on grossly overfitting a large number of parameters to a fairly tiny number of data points. This is the sort of mistake that someone shouldn’t make after making it through their undergraduate stats class. But many of these people are tenured faculty. WTF?

    Like

  7. Political science generally has a predictive power less than that of ju ju. How many political scientists predicted the collapse of the USSR? The only discipline with less predictive power is economics. It is hard enough to accurately predict the past within general parameters. It is impossible to predict the future. Reading around the net it appears there are lots of tenured faculty in the US who are idiots.

    Like

  8. Brad DeLong has a post http://delong.typepad.com/sdj/2012/09/why-is-nate-silvers-forecast-so-much-more-pessismistic-for-obama-than-his-now-cast.html
    That suggests that the current forecast (not the Now-cast) (and I don’t how this changes over time) is weighted 30% economic data, 70% polling data. Presumably such differences could explain a large part of the data.
    I agree with Jay on reading the NY times article — the stats in the PS predictions seemed wildly off. But, that might be OK if your goal is “science” rather than prediction, because you’re building hypotheses about underlying relationships rather than really trying to make a prediction. Overfitting might give you ideas to study, even if it’s poor at predicting.
    Silver is really trying to predict. Weighting heavily towards polling might help with predictions, but it doesn’t particularly help with understanding root causes of outcomes — because they are not a cause They’re an intermediary — well, except for the meta-effect of polls causing changes in voting patterns by having people either choosing to vote, or choosing to support the more popular candidate.
    To compare the PS models, which have a different purpose, with the Silver models, one would have to take out all the polling data and compare apples to apples (the other is maybe like an apple-orange v apple juice comparison, rather than apples v oranges).
    Does Silver show a purely “fundamentals” prediction at the national level? It seems like there are fundamentals predictions at the state level, so he has them somewhere.

    Like

  9. OK, I’m now further confused (the Now-cast is the combination. What’s the forecast then? is it the fundamentals?). Annoyed that the information isn’t more obviously available.

    Like

  10. As a sloght corrective to the characterization of political science predictions, most models don’t say that the current level of economic performance matters; if they did, they would not be predicting an Obama win (as a majority are). Rather forecasting models in political science typically focus on the general economic trend, which is currently positive, and changes in consumer outlooks, which are also improving. These models lean Obama.

    Like


  11. Where do you see that the Now-Cast is the combination? I think the Now-Cast is 100% polls.

    I thought so, too. But, look at the pop-up on “now-cast” for the state-by-state predictions. There, it says, “a combination of the adjusted polling average & state fundamentals, intended to predict the outcome if the election was held today.” The pop-up might be wrong (or some kind of web glitch). And, it contradicts with Brad DeLong’s characterization. So I’m just confused.

    Like

  12. I’m not seeing it. Maybe it’s a Firefox thing. I think part of the deal with being on the NYT is that he has to keep his formula somewhat secret.
    Being able to re-create his work with be more open and scientific, but in general I trust that he knows what he is doing.

    Like

  13. “Being able to re-create his work with be more open and scientific, but in general I trust that he knows what he is doing.”
    I pretty much don’t except to the extent that his predictions are self-policing, as long as he’s not giving explanations, but only predictions. Then, we can tell whether he was right or wrong when we see the outcomes and it doesn’t really matter what his formulas are — if he’s not explaining why he got his results (i.e. the economics, or a particular advert campaign, . . . .).
    He’s specific enough that he’s not susceptible to the nostradamus effects. And, since he presents his predictions in real time, we don’t have to worry that they’ve been adjusted post-hoc to fit outcomes.

    Like

Comments are closed.