Joe Stanley

jakke:

shewalkslikethunder:

scoldylox:

I’ve been trying to come up with a comment on this, and they’re all coming out sarcastic, which is doing it a disservice because it’s a very good piece about a huge methodological problem in polling that a lot of polling firms have been conveniently ignoring for far too long. That is, polling has pretty much always been done via landline. You don’t have think for more than thirty seconds about why that could cause a problem in achieving a random sample of the general public (or of registered or likely voters). There’s a counter-argument that any selection bias gets washed out because the people who are more likely to vote are people who are more likely to have a landline and respond to phone calls on that landline form polling firms, automated or otherwise. That’s kind of a lazy proxy, though, and it sort of hand-waves over the mechanisms for both why people might be more inclined to both have landlines as opposed to cell phones, and be more likely to vote, and usually it ends with a magically conservative leaning poll.

So anyway, if you’re at all interested in polling methodology (and who wouldn’t be?!—no, stop it, Scoldy! I said I wasn’t going to be sarcastic!), Nate Silver compares results with his forecast model when focusing on polls that use cell phone data and polls that don’t. I don’t think the actual results will surprise anyone, but the discussion is really good.

This is important stuff.

But the question when considering cellphone-only voters isn’t “Do cellphone-only voters vote differently from landline voters?” - since pollsters take into account age and gender and race and income and geography and sometimes education and religion, the question is “Does someone with a given set of demographic aspects and no landline vote differently than someone with the exact same demographics but with a landline?” 

To make it concrete, let’s consider a thirty-year-old man who identifies himself as white and makes $45,000 per year and lives in downtown Seattle and never attends religious services. If he doesn’t have a landline, does that make him way more likely to be a hardcore Democrat than if he did have a landline? I’m not convinced there’d be a huge difference. And if there’s no difference, then this cellphone difference doesn’t matter and Nate Silver’s model is overcorrecting based on a very small sample of polls which aren’t really directly comparable anyway.

My understanding from that post, and from similar ones Nate Silver has previously written, is that this difference is not eliminated in demographic balancing by the pollsters.