BOB GARFIELD: This is On the Media. I'm Bob Garfield.

BROOKE GLADSTONE: And I'm Brooke Gladstone. Here’s CNN's Jake Tapper.

[CLIP]:

JAKE TAPPER: If this night ends up being the way that Donald Trump and his advisers hope that it will be, boy, I mean, it’s gonna put the polling industry out of business, [LAUGHS] it’s going to be the voter projection industry out of business.

BROOKE GLADSTONE: Nate Silver is the editor-in-chief of FiveThirtyEight, our partners in election coverage this season, and he was oracular in 2012, correctly forecasting outcomes in all 50 states. Maybe that wasn’t such a good thing, in the long run, encouraging us, especially us journalists, to believe so much, so hard in the purity of poll numbers. Welcome back to OTM.

NATE SILVER: Oh, thank you.

BROOKE GLADSTONE: You’ve said that basically this result was within the margin of normal polling errors. I would say then probably the problem lies with us, those of us who consume the polls every four years, like cocaine-laced M&Ms.

NATE SILVER: Yeah.

BROOKE GLADSTONE: But even you, at 3 am on Wednesday, wrote that you were shocked by the result but not surprised. So why shock and why not surprised?

NATE SILVER: I mean, shocking because of the entire trajectory that American politics has taken over the past almost two years now, since Donald Trump descended the elevator at Trump Tower. I, I think, naïvely, thought that a candidate who made those sorts of appeals that were often based on populism and nationalism and racial appeals, you know, I thought that was not something that a sophisticated country like ours would go for in large enough numbers for him to win the primary, let alone become president. Narrowly, though, polls showed a competitive race and weakness for Clinton in the Electoral College. That’s why we gave Trump about a 30% chance.

Did I, deep in the back of my head think, oh, Trump can really win? I mean, I think I convinced myself that it was possible, based on having had this experience in the primary. I mean, that's what surprised me. I've never gotten so much pushback from the side of the candidate we had favored because we didn't have her favored –

BROOKE GLADSTONE: Enough.

NATE SILVER: - by enough. In retrospect, that seems like a real sign of complacency.

BROOKE GLADSTONE: I don’t know if I'd call it complacency. I’d call it denial –

NATE SILVER: Yeah. [LAUGHS]

BROOKE GLADSTONE: - the same thing that was going on in your head, so you could hold both shock and lack of surprise in your mind at the same time. I’m just wondering what went wrong outside of the heads of journalists. For one thing, you’ve said the state polls were especially bad this year. Why?

NATE SILVER: I think the state polls missed the extent to which polarization along partisan and particularly regional lines was to take place. It was just more polarized. Even within the five boroughs of New York, Clinton beat Trump by a record margin in Manhattan, whereas Trump won in a landslide on Staten Island.

BROOKE GLADSTONE: So you guys have suggested that there were fewer good polls this election cycle. Remind us why that is.

NATE SILVER: Polling is expensive and the key driver of expense is how long it takes you to reach a respondent. So if you call and they answer your call on the first try, that's pretty cheap. If you need to call ten people, then you're spending substantially more time, and the big expense in polling is the labor of people who are making phone calls. If you’re only reaching 8 or 10% of the population, you have to be pretty lucky for the people who respond to be representative of those who don't. And I think what pollsters tend to do is they kind of assume that things are going to be like they've always been. And, therefore, in an election like 2012, where the map changes very little from 2008, the polls can seem to be uncannily accurate, but an election like this one, where you see the coalitions shifting - I mean, it looks like Clinton may have come closer to beating Trump in Texas than in Ohio.

BROOKE GLADSTONE: [LAUGHS] It’s so bizarre. There's also the element of, you know, what you and many other people who study the numbers called “bad polls.” These are partisan polls, online polls, which are still unproven. And yet, many were more reflective of the actual outcome than polls that use proven methodology. Were they not actually bad polls or did they just get lucky?

NATE SILVER: First of all, the online polls, some of them did a little bit better. SurveyMonkey, for example, put out polls in all 50 states that showed something people criticized them for. They showed Michigan and Wisconsin close, but also Arizona and Georgia close. That’s actually kind of in the right direction, at least. And, in some sense, I think we have to root for online polls because the fact is that response rates to telephone polls are going to keep going lower and lower and to have good instead of bad online polls means we’ll have something to wrap our heads around. There are also cheap automated telephone polls that are just kind of - use an auto dialer and kind of spam people.

BROOKE GLADSTONE: Yeah, I always hang up on those.

NATE SILVER: Yeah. Ironically, those did well. Typically, those polls have a Republican bias. They’ll overrate Republicans because they’re only calling land lines.

BROOKE GLADSTONE: Mm-hmm.

NATE SILVER: And this year, I think, that Republican bias kind of canceled out the Clinton bias in other polls -

[BROOKE LAUGHING]

- and so they kind of wound up being right for the wrong reason.

BROOKE GLADSTONE: Your model gave some polls more weight than others, and it considered the economy and history. Given what happened, do you think you may have to do a little recalibration on your model, you know, tinker with the “secret sauce”?

NATE SILVER: No –

[BROOKE LAUGHS]

- because our model did a really good job of understanding the risks in a way that other models didn't, and it did for the reasons that Clinton lost, which is that it recognized that there was a pretty big chance of a split between the popular vote and the Electoral College. It recognized that states are correlated, so when you have the polls go bad in one state, say Michigan, there’s a chance that similar states, like Wisconsin or Pennsylvania, Clinton would lose, as well.

I mean, look, there was no way to look at the polls and to say Trump was a favorite. He wasn't. But to have had a model that was “smart,” quote, unquote, enough to say, you know what, this is a much more competitive race than it might appear and that Trump has a 30% chance, despite trailing in all these polls, to me that reflects a well-designed forecast.

BROOKE GLADSTONE: But while every forecast offers probabilities and not precise predictions, the message from the polling industry was pretty clear - Hillary Clinton was the heavy favorite - which brings us back to the Tapper quote I started with, “If Trump wins, it’s going to put the polling industry out of business.” What do you think?

NATE SILVER: I mean, this is what's frustrating, is that there have been high-profile polling errors in Brexit, and so forth, and, you know, what people have to realize is that if you have a bunch of data points that all say the same thing, they’re all going to be wrong together, potentially, right? So, the fact that there are 100 polls showing Clinton with a three-point lead, it's still just a three-point lead. To me, it's a miracle that the polls aren’t off by more, you know? [LAUGHS] And I bet if you had surveyed members of the Washington, DC press corps and people who cover the campaign, they would have made Clinton a much heavier favorite than 70/30.

BROOKE GLADSTONE: Yes, because they’re just human beings and –

NATE SILVER: I know but this is [LAUGHS] –

BROOKE GLADSTONE: - cloistered in their little coastal environments.

NATE SILVER: And we are too.

BROOKE GLADSTONE: But we want to trust the numbers, and this shakes our trust, no?

NATE SILVER: The purpose of a model or a forecaster or a journalist is to do two things, right, which is to say, here is the evidence and here is why the evidence might be wrong. FiveThirtyEight, I think, was one of the only sites out there who were saying, here's the polls, yes, Clinton's ahead. Here's why they might be wrong and, if they're wrong only a little bit, Trump can win the election. If people aren’t being careful about understanding that a three-point race is a close race, then they’re getting bad information. To me, it's a little bit bizarre for people to be blaming the polls when, number one, at least our analysis of the polls suggested that Clinton's lead was not safe but, more broadly, when the conventional wisdom was way worse –

BROOKE GLADSTONE: Mm-hmm –

NATE SILVER: - and was way more dismissive of any chance that Trump had. People should check [LAUGHS] their own assumptions and the way they cover the campaign. Even if you're not making predictions, you know, you can see which reporters thought Trump actually had a chance and which reporters thought Trump was kind of a joke that would expire at 9 pm or so on November 8^th.

BROOKE GLADSTONE: [LAUGHS] Nate Silver is the editor-in-chief of FiveThirtyEight, our partner during this election season. Nate, thank you so much.

NATE SILVER: Thank you.