Can We Trust the Polls?
BROOKE GLADSTONE This is On the Media, I'm Brooke Gladstone.
BOB GARFIELD And I'm Bob Garfield. Nate Silver, founder and editor in chief of FiveThirtyEight.com. Never claimed to be an oracle, but after his precient 2012 forecast of an Obama victory, he was instantly anointed. Here's Jon Stewart.
[CLIP]
JON STEWART Nate Silver, lord and God of the algorithm. [END CLIP]
BOB GARFIELD Four years later, the Oracling racket got a little harder. The 538 model, based on an amalgam of polls across the country, predicted a 70 percent chance of Hillary Clinton winning. Which was probability, like a 70 percent chance for a cloudless day, but what hit was in the 30 percent. A violent thunderstorm and Silver has spent four years explaining that the forecast itself wasn't wrong. That's because the public, especially the Democratic public, had interpreted the 70-30 ratio as a near certainty. And in their disappointment, they blamed the polls.
[CLIP]
NEWS REPORT There's a lot of distrust when it comes to polling out there, after all. Donald Trump did win the election. [END CLIP]
BOB GARFIELD Well, here we go again. Silver has adjusted his methodology based mainly on unique circumstances, but the model remains the model. Nate, welcome back to OTM.
NATE SILVER Hey, how are you?
BOB GARFIELD Splendid, thank you. So in 2016, when Trump beat the odds, a lot of people wanted to run you out of Dodge.
NATE SILVER Yeah.
BOB GARFIELD You have stood by the model, but also tinkered at the margins - in what ways?
NATE SILVER Well, I would say that our forecast did a great job in 2016, because there are very few people that gave Trump a 30 percent chance. And 30 percent is kind of a lot. It's like a good Major League Baseball player getting a base hit. So I don't think there's anything about our model that would intrinsically have necessitated change 2016, except we do have a global pandemic. We have uncertainty about how people are voting where many people are voting by mail for the first time. We have some of the worst economic data that we've had in the history of the country. We have all types of crazy news stories developing. We had a Supreme Court justice die, another one being nominated by the president himself get COVID. So, our model does try to account for the extra uncertainty in this election. And as a result, it winds up being a bit more conservative relative to how it might have been in the more, quote, unquote, normal year.
BOB GARFIELD A lot of the grief in 2016 had less to do with your forecast than partisan expectations and polling illiteracy. What are you doing in 2020 to manage public and especially media understanding and expectations?
NATE SILVER One thing I think is we don't foreground the probabilities as much. Right, you can see them. Right. But if you actually click on 538, the forecast, you see a bunch of different maps simulating different worlds that the election could turn out to be. And there's verbiage. Then you have this kind of scroll down to see the numbers. Right. Because one thing about numbers is once you put a number in front of somebody, then they kind of stop seeing anything else or thinking about anything else. We want to provide the context for people and for them to understand where the uncertainty comes from, something about what we're simulating, what we're not. It's important, by the way, to note that, like, our model does make some assumptions. And one important assumption is that all votes are counted and that the Electoral College respects the popular vote in each state. And those things are not certainties. So there's an attempt to steal the election that would be outside the scope of our model.
BOB GARFIELD What else are you doing to change your presentation?
NATE SILVER Part of it is the messaging that we have when we're doing media interviews like this or going on Twitter. Right. It's kind of a matter of tonally how people interpret things. In some ways, though, it's a little easier after 2016 that people have a visceral experience with the lower number coming up. And so people don't need as much persuasion as they might have to take stuff seriously.
BOB GARFIELD First time. Shame on you. Second time. Shame on me.
NATE SILVER Yeah. So one thing about sports fans, right. Sports fans don't tend to get mad at you if your model says oh, there iss only a 13 percent chance that the New York Giants win this football game and they win. Right. Because they see enough games where they seem 30 percent chances come up all the time. And I have you down to one percent or something. You know, the Atlanta Falcons blowing a Super Bowl, it might be different. But people are used to dealing with probabilities when it affects things they have to deal with every day. When it seems like something abstract that maybe they don't grok it as much, but we're trying to de-abstract what we're saying, I guess. Right. And make people understand that, like, look, even if Trump's chances go down to five percent. Well, five percent is fairly high when you're talking about potentially a world changing and a life changing outcome.
BOB GARFIELD We are speaking on Thursday, October 15th, with less than three weeks to Election Day. What does your data forecast for the presidential race?
NATE SILVER So as of this very moment, it has Trump with a 13 percent chance of winning Electoral College and Biden with an 87 percent chance. So it's about one in seven. Let's have you round it up to. Six for Trump. You can kind of think of it as a game of Russian roulette. Where one out of six times, the chamber comes up where Trump would win. So he's far from out of the running. At the same time, there's not really a precedent for a candidate overcoming this large a deficit this late in the race. You can maybe go back to Dewey versus Truman in 1948, where Dewey was ahead by about five points over Truman and Truman won. In this case, Trump's down by more than that. So mathematically, our model thinks it's possible that he could win, but we're running out of historical precedents that are favorable for him. So in some sense, I think given how light Biden's lead in the polls is, I tell people to look at 13 percent and say, hey, that's not a sure thing. At the same time, this is not one of the more competitive elections that we've seen, at least based on if the polls are right, obviously.
BOB GARFIELD To use the football analogy The New York Times embraced last time around: Biden is looking at a very short field goal, but as you said, these are unprecedented times. What could happen between now and November 3rd to make him to miss that little chip shot?
NATE SILVER Well, this is, in some sense, where it gets harder, right? It's kind of one reason why I like to stick to the mathematical modeling is like it's hard for me to come up with a narrative by which Trump comes back. But there are a couple of things. Maybe some big pharma company announces progress on a COVID vaccine. You won't get people vaccinated, but that might make people feel a bit more optimistic. Maybe there's a stimulus package that gets some money in voters hands before the election, again, that's looking a bit less likely. But I should say, by the way, one crazing that could happen is that what if one, the Democrats were to get COVID. There are people in Kamala Harris orbit who were diagnosed with COVID. They are doing campaign events, too, and so that's something that could happen, but it's hard to know. I mean, I think Trump seems to be doubling down on all the strategies that put him 10 points behind to begin with. And so instead of pivoting, he's repeating a lot of very unpopular messaging on COVID, for example. Actually, most voters favor precautions on COVID. There's a myth of Trump as being as masterful campaigner that I don't know has really borne out. I think it was very clever in how he won the GOP primary, but he just barely beat Hillary Clinton. His party lost a ton of seats in the House in the midterms in 2018. And now, if polls are right, we keep using that phrase. He's in a lot of trouble in 2020. And so I'm not so sure that Trump's instincts are as good as sometimes they're made out to be.
BOB GARFIELD You mentioned that football fans have somehow managed to internalize on any given Sunday and, you know, maybe the New York Giants could trounce the Kansas City Chiefs. Is it a good thing that people have learned that the polls are not an absolute prediction?
NATE SILVER Yeah, I mean, they never were an absolute prediction. Polls have not gotten any less accurate or any more active, for that matter, over the long run. And Clinton lost despite only a fairly normal small polling error because she was not that far ahead in the first place. So, yeah, people, I hope, would internalize the lesson that if a candidate is only a couple of points ahead in the polls, then they're going to lose not half the time, less than half the time, but fairly often. Again, this race is different in that Biden is more than a couple of points ahead. You'd have to have maybe twice the polling error that you had in 2016 for Biden to lose, and that can happen, but now we're getting a little bit more out on the tail of the probability distribution. But polls are always imperfect instruments for sure.
BOB GARFIELD Look, I am obliged to observe that after 2016, first you got dumped on for somehow misleading the entire world about Hillary Clinton's chances. Then when you did interviews saying approximately what you have said today, you got dumped on for being a defensive jerk. I think, as you put it. The conversation we're having now, is this a way of inoculating you and 538 against, you know, whatever outcome we should see on November 3rd.
NATE SILVER It's not a strategy at all? It's just me being honest. You know, it was it Mark Twain that whatever you said, if you're just honest, you don't have to worry about kind of saying the wrong thing? The honest truth is that Joe Biden is way, way ahead in polls. I don't connect those polls, but I think it's pretty reasonable to think that given where the country is at, that Joe Biden would have a lead given things people are going through. And how many people have died of of COVID. Empirically, could the polls be wrong enough for Trump to win? Yes, it's not likely, but it's it's hardly unfathomable. So, you know, I have to give people the best information that I can. And I would love to hedge more. I'd love to say, oh, it's 60-40. We don't know who is going to win, right. But that wouldn't be honest and that wouldn't be responsible given what the data showed. So in the long run, you know, Biden should win seven out of eight times, you know. I know we'll probably get more crap if the one out of eight times comes up, but I can't do anything about that. I think it's the right forecast.
BOB GARFIELD Nate, thank you so much.
NATE SILVER Definitely. Thank you.
BOB GARFIELD Nate Silver is the founder and editor of FiveThirtyEight.com and author of The Signal and the Noise: Why So Many Predictions Fail, But Some Don't.
I just want to point out that I was never mad at you. I understand how the weather forecast works.
NATE SILVER [LAUGHS] I appreciate it. It was a good interview, though.
BROOKE GLADSTONE Coming up, an app that offers a haven for the convenient recruitment of online militias.
BOB GARFIELD This is On the Media.
Copyright © 2020 New York Public Radio. All rights reserved. Visit our website terms of use at www.wnyc.org for further information.
New York Public Radio transcripts are created on a rush deadline, often by contractors. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of New York Public Radio’s programming is the audio record.