Fake Polls: NBC/WSJ Pollster Peter Hart
February 7, 2017
BOB SCHIEFFER: With us this time, Peter Hart, the dean of American pollsters. He founded Hart Research in 1971, and since 1989 has run the NBC/Wall Street Journal poll. He’s polled for 55 senators, 40 governors over the years, and such institutions as the Smithsonian and the Bill and Melinda Gates Foundation. National Journal named him one of the 150 most influential people in America in shaping national policy. Peter, welcome.
PETER HART: Thanks, Bob.
MR. SCHIEFFER: So I thought we’d just go ahead and start with the bad news: pollsters took a real beating in 2016. The Trump people said polling consistently misunderstood the anger that propelled their candidate to the White House. Others noted you just didn’t foresee Trump’s victory. Is that criticism justified?
MR. HART: Sure. Why is it justified? Because if you look at most people who talked about this election, the told you that Hillary Clinton would be – would win. But there are two sides to this. The first side is, if you look at it only in statistical terms, take the NBC/Wall Street Journal poll, or the one you’re associated with, CBS and The New York Times, we both showed Hillary Clinton ahead by four points in our national poll. But what was the answer? We were only measuring the popular vote, and indeed, Hillary Clinton won the popular vote by two points. So you say, how bad did the polling industry do? Not so bad on a national level. But in terms of the state polls, boy, there were a lot of wide mistakes there. And if you were a voter and you were out there watching this, you would have said, why can’t the polling industry get it right? Because one man won, one person lost, and it was reported almost the opposite. I’ll talk about my mistakes. I think I made three major mistakes in terms of interpreting the election. The first major mistake I really made is that I underestimated how much integrity counted in this election. We knew from the absolute beginning people did not trust Hillary Clinton. We measured it in 2007, before she ran against Barack Obama. We measured it in 2015 and 2016. And at the end of the campaign, essentially what we found out is essentially 22 percent of the voters had positive feelings about her when it came to integrity and 56 percent of the voters did not. Integrity is always the central factor, and I think we overlooked it because we said, Donald Trump is the centerpiece of this campaign. I think, in the end, it was Hillary Clinton. Second thing that we missed, as much as anything else, is we didn’t measure the Rust Belt. And when I say we didn’t measure it, obviously the state polls do. And the national polls, if you look at an NBC/Wall Street Journal poll – and we’ve been doing it for 27 years now – we look at the East, we look at the Midwest, we look at the South, and we look at the West. And what happened is we didn’t say, you know, this is going to run through the Rust Belt, and instead we didn’t look at that – those groups. We didn’t look at Michigan and Wisconsin and Ohio and Pennsylvania as an individual thing. When I went back – after it was all over we went and looked at it, and guess what? Trump was winning in the last poll, and we didn’t pick it up and we didn’t have it. The third thing, most important element in this, is 18 to 20 percent of the voters voted with negative feelings towards both Hillary Clinton and Donald Trump. And we had been looking at that early on, and their vote was split, that 18 percent. And it was only at the end that we started to see that Donald Trump was moving up, Hillary Clinton was losing altitude. And that vote on election night came in 47 to 30 (percent) for Donald Trump. Three major sort of errors in looking at this. And I hold myself responsible; I don’t hold the industry responsible. But those were all things that – signs that we could have picked up.
MR. SCHIEFFER: Let me just kind of go through this one at a time, because there are a lot of things to unpack here. But I want to go back to your first reason, that people just didn’t trust Hillary Clinton. Are you saying that she lost it rather than he won it?
MR. HART: I would say yes, of course she lost it, because essentially her problem is that she never corrected her deficiencies. And instead of being unpacked, they got packed on because whether it was the speeches from Wall Street or the emails or the Comey report, all of those things kept reinforcing one thing: you can’t trust Hillary Clinton. And that’s what – that’s what Donald Trump did throughout the campaign.
MR. SCHIEFFER: So basically what you’re saying here is that people didn’t trust Hillary Clinton to start, and as the campaign unfolded they found more things to buttress the original opinion that they had.
MR. HART: Exactly right. And my problem was that Hillary Clinton had certain goals that she needed to obtain. One was to make herself more likeable, second was to be able to identify with the average working person, and the third thing was to be able to give people some sense of what her values were. And the way she explained these things were always in issue terms or in some other fashion. She never opened up. The thing I always felt in this is she was trying to break the glass ceiling, become the first woman president. The voters were trying to break a glass curtain and trying to understand her. They never did. They didn’t vote for her.
MR. SCHIEFFER: Now I want to get something else you talked about, and that is the state polls.
MR. HART: Yeah.
MR. SCHIEFFER: You know and I know there are polls, and there are polls.
MR. HART: Right.
MR. SCHIEFFER: Were so many of these state polls that we heard about and reported on, was the methodology just not up to the standards that it should have been?
MR. HART: Sure. I mean, essentially, we don’t have anybody overseeing the polling industry. And so any college, any university says, hey, this is going to be great publicity, you know, we’ll be the state college poll of blank. And some of them are quite good and they’ve had a long reputation; others just showed up. And we don’t make any differentiation between good and bad, and whether it’s done online, whether it’s done on telephone, whether it is done as an intercept poll. All we say is the margin of error is such and such. Can you explain, Andrew, what the margin of error is to me? You can’t.
H. ANDREW SCHWARTZ: It’s hard. (Laughs.)
MR. HART: Yeah. You can’t because you’ve heard a little spiel that says it’s plus or minus 3 percent. Well, I’m sorry, if you’re doing an online poll, there is no margin of error – sorry – because it’s not a universe that they’re drawing from. And each of these things, for the public, we treat it all as equal. And let me take you a step further. We have something called Real Clear Politics, so we average all the polls. Well, Bob, I know you’re a wine drinker. Would you take a dollar bottle of wine and put it in with a $300 bottle of wine and say, hey, let’s see how the average of the wine is? You’d say, no, that’s an awful idea. Why is it any better for polling to take a zillion polls, some good, some bad, and put it all together?
MR. SCHIEFFER: You know, I’m not a drinker, but even if I were I would know better than that.
MR. HART: I stand corrected. (Laughter.)
MR. SCHIEFFER: But, you know, you talk about margin of error. Here was my take on that. During this campaign, I kept hearing people say so-and-so is leading by one point in the soand-so state poll, or somebody else is leading by
two points. Well, you know and I know there is no one-point or two-point lead. That’s always in the margin of error. And too many times, I think, we never heard a margin of error mentioned.
MR. HART: Well, I think they mention it, and I would say that is one advancement in covering polls. But the difficulty that we have is they’ll mention it, but they only mention there’s a one-point differential. In reality, it may be 44 to 43 (percent), but that means candidate A could be somewhere between 40 and 46 (percent), candidate B could be between 47 and 41 (percent), and so the one-point hides that element and people get mystified. I don’t blame people for not trusting polls. It’s hard to understand.
MR. SCHIEFFER: You know, one of the things that I try to point out in defense of pollsters –
MR. HART: Thank you, Bob.
MR. SCHIEFFER: – is it’s much more difficult now to poll than it used to be. The coming of the cellphone has changed the world of polling.
MR. HART: Totally, that essentially what we have is at this stage of the game, if you take an NBC/Wall Street Journal sample, 45 percent of all of our phone calls are made to those who have cell only – I’m saying cell only. And it’s that important because that’s how much the universe has changed. And it is much harder to get respondents. We get under 10 percent of respondents willing to take a poll. Well, how about the other 91 percent that we’re leaving out? And I think all of these things has made it much harder. And I think the biggest thing – and I hope we get into it – is we’ve stopped listening to the voices of the voters and we’ve started just making it as statistical or dynamics and analytics. It just doesn’t work.
MR. SCHIEFFER: Let me just ask – let me just underline that with one thing I know about the CBS poll. In the old days, when we’d do 1,500 people for a nationwide random poll, our pollsters tell me we would call 3,000 people to get 1,500. They tell me now that in order to get 1,500 people they have to call 30,000 people. And I don’t care how you weight it, Peter, you still have to wonder who are the 1,500 people you wind up with when you have called 30,000 people.
MR. HART: Well, that’s exactly – I mean, that essentially we’ve got major problems facing the industry, and the most important is respondent willingness. And we just don’t have it as we did before because it used to be a privilege to be interviewed, now it’s a bother.
MR. SCHWARTZ: Peter, you know, it seems now that everybody has their own narrative depending on what their political point of view is. Have you ever seen the country be more divided than it is now?
MR. HART: It is exceptionally divided, and it’s divided because essentially we’ve chosen up teams. We used to listen to the CBS Evening News, the NBC Evening News or the ABC Evening News. And we were there to get the information, and while each of them may have had different stories they emphasized, today we have rooting sections. There’s the red team and the blue team. And whether it is print or online or on electronic, you’re on your team. And at that stage of the game, you’re no longer getting that sort of cross-section. And we sort of pretend as though it is, but at this stage it’s so divided, and because it’s divided the ways in which we reach decisions and the way in which we understand public policy is so different because it’s not only false news, it is – it is news with a slant on every basis. And that’s a problem. And that’s what we try and measure, which is accurate public opinion. But you look at the polls, and so many cases you say this isn’t a balanced question; it’s just not accurate. And yet, some news organization takes it and puts it up at the head of – headlines, and that’s what we end up with.
MR. SCHWARTZ: Do you think we’re listening enough to the American people and what they have to say?
MR. HART: We’ve lost it. I mean, last year I did for the Annenberg Center for Public Policy about 10 different focus groups across the country and shared them with you. And when you stop and listen to the voters, we understood this election from beginning to end. I got fooled by numbers, but I never got fooled by the voices of the voters because this election was crystal clear from the beginning. The voters never wanted Jeb Bush and they never wanted Hillary Clinton. It was about change. It was about the horrors in their lives and where they were coming from. And whether it was the absolute beginning or the end, it was the same campaign, and you could see it and you could feel it. And while everybody said Donald Trump is sinking, he’s gone, he’s gone, he’s gone, every one of the reports I kept writing said, no, don’t miss this, and understand what’s happening here. And one of the things that was fascinating is my first poll – or focus group that I did, in Colorado – Aurora, Colorado – I found a woman who essentially almost became the quintessential voter. And she told us about the life and the struggles and the economics and everything else, and she was a strict, hard Republican, January of 2015. And I said to her, so, if you could hear from any one politician – Trump hadn’t announced, you know, we didn’t have all of this, Bernie Sanders – she said what I’d like to hear from is Elizabeth Warren. And I went, whoa. And essentially what she was telling us about was the rebellion against what was going on. And we tend to think of this as something that’s just got manufactured. If you live these people’s life and you see what they’re going through, essentially you can see it. We thought Donald Trump was something unusual. All he is is amateur professionals. And that’s Airbnb, it’s Uber, it’s all the other things that are happening. We thought the establishment was the thing. No. Bush was through. Clinton was through. It was not about money. It was SpaceX. It’s Elon Musk. It is Amazon. All of these things are changing in our society, and not recognize the – Dan McGinn, who I work with, so keen and so aware of all of this, and somehow we sort of put it over here. They’re all connected.
MR. SCHIEFFER: How is it that Donald Trump was able to just – these charges, these things that I said over and over, every time he did one of these things – you know, what he said about Megyn Kelly, what he said about John McCain – I would say, well, that’s it, he has now crossed the line. And yet, it seemed to have no impact whatsoever on the voters who were for him.
MR. HART: Yeah. Two things. Going back to the themes of 2016, the rise of incivility in our society. We’re pretending as though somehow Donald Trump was exempt from that. No. It all started going back to Jerry Springer 25 years ago, and it moved forward. And we look about the personal things that are on the internet, et cetera. We have become inured to so many of the things that are out there. So, in the end of the day, what happened is that essentially the voters said, you know something, he’s repulsive, I don’t like it, but I like what he wants to do. He caught the zeitgeist of the country. And at the end, it was a very hard vote for voters, that they didn’t say, gee, I’m rushing to him. And essentially, you ended up with so many of the people voting negatively more than positively.
MR. SCHWARTZ: Well, there’s a lot of confusion because Donald Trump gives a speech, an inaugural speech, where he paints a very bleak picture of the United States. And what you’re saying is, is there’s a lot of hurt out there. A lot of Americans think, well, you know, things aren’t so bad. Barack Obama told us things weren’t so bad. What’s the truth? Are there two Americas that we’re just missing?
MR. HART: No, there’s one America, and the one America is there are lots and lots and lots of people hurting. But against that, there is a sense that we want to change the dynamic, we want to change the way things are. And it’s not that there is not advancement in our society. But the question for the – for the Trump administration is going to be balance. Right now, it’s totally unbalanced. And it’s not only that they’re playing to their base, but the fact of the matter is they’re taking institutions and unbalancing it. And it’s going to be, whether it’s the Court or the Congress or all of these things, the public is going to end up weighing this. And the interesting thing is the Trump administration has come in, and the president, as the most polarized person that we’ve ever seen. And he says, don’t look at the polls. All I’m telling you is go out and listen to the voters. They’re disturbed. They’re worried. They don’t want tweets. They want some sense of how we’re going to put this nation back together. And it’s not a liberal agenda, it’s not a conservative agenda, it is a sense who’s going to listen to us and represent us. The Congress is now very much on the line, and it will be interesting to see how they handle it.
MR. SCHWARTZ: How much rope does he get, I mean, if he doesn’t deliver, if he doesn’t, you know, become that can-do, get-the-job-done person? Certainly, you know, as Bob just said, he’s had a pass on a lot of things. You know, people – how long can people say, well, there’s a – he’s only been in office a couple weeks, but how long can he, you know, be on a learning curve? Or how long can he be immune to criticism that sticks to him?
MR. HART: Well, there are two sides of this. The most important thing is he has an exceptionally long leash with his own voters. They want him to succeed. They voted for him for a reason. But against that, it is not that they’ve endorsed, one, his behavior; or, two, every policy. And his ability to understand the difference between the two. For example, those people who voted for him but have negative views about him and Hillary Clinton – so it was as an alternative – they have no interest in him building the wall. So if that’s a central element of his administration, he’s not talking to those people. The sense in terms of the economy, if it goes in the right direction and it’s building, the leash will be that much longer. But I think you come back to the big and central questions, and the big and central questions will really become a sense of are we getting together and is there balance. And that’s what I think is at the forefront.
MR. SCHIEFFER: I want to go back to the campaign and talking about Hillary Clinton. Her people would argue that the reason she lost is because Comey, the FBI director, announced he was reopening the investigation into the emails in the last weeks of the campaign. Was that a factor?
MR. HART: Huge factor, but there are two ways to look at it. If you look, the fascinating thing is in the Rust Belt the switch in voting attitudes in Wisconsin and Michigan, of those people who decided in the end, went overwhelmingly – overwhelmingly – to Donald Trump. And so you can say that was a central factor and it was Comey. But the other side of this is it wasn’t something that, oh my God, where did this come from, we never saw it coming. Well, the answer is any poll tells you exactly what was happening. The Clinton campaign stopped any individual state polling at the beginning of October. And they say, well, yeah, but we had analytics. Well, Bob, you follow baseball. Analytics will tell you one thing, but they don’t tell you what’s in people’s hearts and how they’re going to go. From my point of view, they never heard the voter. They could have dealt with the integrity issue, and there were lots of ways they could have made her sympathetic. If I’d been in charge of Hillary Clinton’s campaign, look, I would have demonstrated who I’m fighting for and who I’m fighting against. And when the problems with EpiPen happened, I would have taken Hillary Clinton and put her out in front of that headquarters and say these are people we are going to – we are going to hold accountable, and we are going to change things. And when the – Wells Fargo came out with their problems and their integrity, I would have put her right in front of there and said these banks are going to pay a price, and they are going to either behave in a(n) ethical fashion or we’re going to – we’re going to take them to court and finish them. And she never gave you any sense – she gave you six-point programs and four-point programs, but she never connected with the voters.
MR. SCHIEFFER: Do you think that another Democrat could have won? MR. HART: Not one that we naturally think of. In other words, if you said to me, could Bernie Sanders have won? No. I just think his – too many of his ideas were outside the mainstream. Did he hit the right zeitgeist just like Donald Trump? He was absolutely dead on in terms of that. I think that he would have had a hard time with his record being dissected. Joe Biden, perfect in terms of temperament, in terms of tone, in terms of where people want – but it was an election of change. I think it would have been a tough one for Joe Biden to win. But I think, at the same time, Joe Biden would have changed the dynamics. I would have been a very different campaign. But it was a change election.
MR. SCHIEFFER: It sounds like that Barbara Bush may have known what she was talking about when she said, I guess in 2015, the country is just tired of Bushes, Kennedys and Clintons.
MR. HART: Yeah, it was. I mean, essentially, we’re unhappy with where we are as a country. We’re unhappy with the people in power, the status quo. It was a change election.
MR. SCHIEFFER: Let me just ask you one more question, because just the thought occurred to me as I was listening to you here. How is it that we wound up this time, for the first time that I can ever remember, that – with two candidates that the American people neither trusted nor liked? That’s not about polling. That’s about our American electoral system. How did that happen?
MR. HART: Essentially, one nomination was rigged and the other came through a very populist manner. I mean, the Democratic nomination could have been very different if it had been a more open system, and I think that Hillary Clinton would have been a better candidate. For Donald Trump, give him 11 other candidates and he always was the flame, and all the moths were drawn to the flame – couldn’t help it.
MR. SCHWARTZ: So what happens now with the Democrats? I mean, you have a lot of elements out there of progressive groups taking on tactics like the Tea Party took on. You have a Democratic Party, probably an enormous field of candidates emerging in the next two years. What does the Democratic Party have to do to find itself?
MR. HART: Well, I think the Democratic Party first and foremost has to go back to the roots. Its roots have always been in the middle class, and I think that they’ve stopped listening and talking to the middle class. They’ve been legislating for the middle class, but they haven’t identified or been there. The second thing is that they need to understand that they have to get back to the state legislatures and those local races, and build from there. And the third thing – and people don’t recognize this – is history is in its favor. The groups that are growing are Hispanics, minorities, young people, Millennials, college-educated. Those are the groups the Democrats have done well with, and the answer is there is going to be a rising coalition. We always think it’s absolutely the darkest and the Democrats will never find their way, and along comes a Bill Clinton, and then he becomes the beginning of an era.
MR. SCHIEFFER: You know, I’m not convinced that the Republican Party is in very much better shape right now than the – than the Democratic Party is. And I would take it back even further than you would and say I think what’s happened here is that the whole process of running for public office has become so odious that more and more serious, good people are simply saying I want nothing to do with it, there are better ways I can get more done, I can accomplish more even in public service than I can by running for public office. I think the whole thing has to be turned upside down.
MR. HART: Well, I think you’re very much on target, because that’s where we’ve been. That’s what the voters have been trying to say. And obviously, until public service is something that is respected and valued, you’re not going to get the best people coming into it. And the same thing, the question of ethics. It’s a huge question for the Trump administration, but the reverse is you have to make it attractive and acceptable to have people who are quality people that don’t get left out.
MR. SCHIEFFER: Peter Hart, who knows more about polling than any single person I know. And I’ve done some polls on that. (Laughter.)
MR. HART: Thank you. Thank you, Bob. Thank you, Andrew.
MR. SCHIEFFER: Peter, it’s great to have you.
MR. HART: I’m honored.
MR. SCHIEFFER: For Andrew Schwartz, Bob Schieffer. Thanks for listening.