The Importance of Polling in a Democracy with John Zogby and Lucca Ruggieri | Constitution Day 2023

0:01 [Music] in Federalist 1 Hubley is placed before the people of the United States a challenge he states that it has been reserved to the people by their conduct and example the opportunity to decide on an important question whether or not societies of men are really capable or

0:21 not of establishing good government from reflection and choice or whether they are forever destined to depend on their political constitutions for their political constitutions on accident and force people of the United States had a responsibility to meaningfully reflect and carefully decide whether this new

0:42 constitution would adequately protect their liberties we have that same responsibility today as we work to live under that Constitution continue to operate a society that serves for everyone’s life liberty and pursuit of happiness we’re pleased to have a conversation today with two people who are playing a role in that conversation we’re joined

1:04 by two pollsters one who is more seasoned and one who is just getting started John zogby has been a pollster and opinion researcher for decades has an excellent track record on a variety of questions touching many aspects of American politics and daily life Luca ruggieri has just been in this business officially for about a year but has

1:26 already done some great work and we’re excited to talk with both of these gentlemen about why they chose to do this what they think their work does to to contribute to our national conversation and so if we can do so I welcome John and Luca glad to have you with us today you’ve been at this for a while could you just tell our audience briefly what Drew you to polling and how

1:47 you got started with the firm that you’re running now there were three career paths in the 70s and early 80s for me I was a college professor I was a community activist in including actual Community organizer president and uh I was also uh uh someone who was

2:09 was fascinated by the early 80s I got a chance to run for mayor of my community and to do my own polls and uh I guess to quote Thomas Marshall I’d rather be white president I lost as mayor but I knew it was exactly how much I was going to lose by and I thought hey no this is pretty good so in 1984 uh I started

2:32 playing from the bottom up worked my way up to ultimately Murdoch Empire Reuters NBC News Luca you’re just getting started I take it you’ve not yet been a college professor tell us a little bit about your story and how you came to start Patriot polling yeah well I’m not even in college yet uh but the way that it started is that um I uh a buddy and I

2:55 who I uh started Patriot polling with uh his name’s are Hong Kong uh we were on the train and I’m really more into the political side and he’s into uh data science and that sort of stuff and we were talking um this is uh the summer of 2022 about the polling and how we thought you know these midterm polls are going to be terrible the polls in 2020 were terrible the polls in 2016 were terrible what is

3:17 the differentiating factor there that separates the good pulse from the bad polls and you know we’re all thinking about uh what are we going to do to get ourselves into college so I had the idea you know it well why don’t we try conducting a few of our own polls I mean I I’ve got the political side down you’ve got data science down so you know what’s the worst that could happen and then uh before you knew it uh we had

3:37 Nate silver in 538 uh picking up our polls and we were I think the first uh polling company to be founded by high schoolers to be put on the 538 Aggregate and you know once that happened you know really got the ball rolling and it turned into something a lot bigger than we thought it would but it’s been a terrific experience sounds great well but I think every entrepreneur could tell you a similar

3:57 story right John you start something and it goes lots of places you didn’t think it would a lot more work too oh exactly I got a D in statistics by freshman year in college I love to tell as well could each of you take just a minute and reflect on the role of pulling in American politics and John I’m going to start with you and then we’ll go to Luca as soon as you’re finished

4:18 um it’s organic to a democracy a democracy is based on all of the people of I am which means that his policies are made or debated essential ingredients not good dominant and all polling does is offer a scientific approach to

4:40 obtaining what that public place is you know there have been other methods years in other forms of government as well as collecting public opinion we add some science Luca I agree with what John said but I also think that polling does have some influence on the way that things turn out I can just say you know uh in the

5:01 senate race in my home state of Pennsylvania um about a year ago uh in the Republican primary which is really what what uh got me uh really interested in polling uh the the polls original uh they they showed uh originally Dave McCormick and uh Dr Oz in a virtual tie but um towards the end of the race uh this uh you know

5:23 sort of trump candidate Kathy Barnett she started uh rising in the polls and a lot of um McCormick supporters uh switched their vote to Oz uh because they thought that Barnett was going to be it was going to be a race between Barnett and Oz and in the end it ended up being Oz that one by less than a thousand votes so I do think polling can have uh a material impact on on

5:45 elections and and a lot of policies are decided based on polling and but I do think it’s important uh pulling so there’s a very important important role in giving both uh like an officials in the public an insight into what the uh General uh National opinion is on on certain issues or local opinions or whatever it may be

6:05 but it serves a very uh important role in our democracy John is there attention then if if what Luke is describing makes sense is there any kind of tension between poll polling and pollsters and individual choice is it is it how does it help or um does it put thumb on the scales how does how does that happen is that how

6:27 would you describe that dramatic it’s not that dramatic and I’m not so sure um that what Luca says is right um you know they the dominance or the heavy influence look you didn’t need to pull uh to give you a sense that Richard Nixon was going to destroy George McGovern there are people out there who

6:49 say well you know this guy doesn’t have a chance I’m going to vote for the underdog there’s so many factors that are in how and why people make a decision I’m not so sure that this is among those dominant actors do I like the candidate does he or she present themselves well do they represent my core values

7:11 um those are important things couples can tell us you know answers to those questions let’s let’s get to the the craft of polling itself Luca how did you put together questions as you were building the polls that you did over the past year what what was a good question how did you decide what to ask how did you decide how to ask it

7:33 we just asked questions the way they were going to come up on the ballot because I think if you add anything else in there uh it there’s a lot of personal bias that starts get injected so you know you just go to the ballot what we would do um the whole reason why we were able to pull is because we used this method called ivr which is uh basically a robot uh calling a bunch of uh people and

7:54 reading out questions and what it would do it would ask you uh you know in the uh Georgia Senate race are you voting for uh Rafael Warnock the Democrat or Herschel Walker the Republican and that would be it uh and there would be and then there will be the undecided option but we didn’t try to add any sort of uh other uh descriptors uh to our polls

8:14 because we think uh that’s the way it’s going to come up on the ballot and that’s the most accurate way of uh prefacing it John I know you’ve done a lot more polls as you’ve done so what what else have you found goes into the the preparation of questions I think first of all it’s very important as Luca mentions to um to include the name as it’s going to appear

8:35 on the ballot that’s really consistent with ultimately how people make their decision but then I also uh we as a company have never really relied simply on the horse race I don’t know what people’s values are so whenever we’ll on a candidate or policy will ask our

8:55 questions for a tune to uh how does this best Express who you are which of the following best describes how you feel or where you stand we’ll try to get a core values you feel agree or disagree that terminating fetus is manslaughter right do you believe on the issue of abortion

9:18 ultimately a woman has the right to choose oftentimes on that question all the time you will get crossover look at people who actually feel that abortion is manslaughter and that a woman has a right to choose that allows us the Suffocation to know which candidate is ultimately addressing those voters who

9:39 are going to be John can you give me a maybe a story that you’ve encountered in your work where the values told you something that maybe you didn’t expect to find or that turned out to be more deeply revelatory of where a question was headed I’m going to give you a famous story uh that I’ve written about and I’ll try to

10:01 keep it short but in Daily tracking for NBC News and Reuters presidential race in 2000 we had Gore going up ultimately Gore did win the popular vote as you know we’re the only ones that captured or by one half of one percent uh you know the popular vote

10:22 towards W question but I added a question in um and it may seem frivolous but I told I said to voters live in the Land of Oz and this year there’s an election for mayor of Oz one of the candidates is the Tin Man it’s all brains and no heart and another

10:43 of the candidates is a scarecrow he’s all heart and no brains would you vote for reflection were held today and it came out 46.2 for the Tin Man and 46.2 for the Scarecrow that told me an awful lot about people’s values the fact that probably weren’t going to know who won

11:04 that election and as you know we didn’t know who won that election until the Supreme Court died I remember that story right down to the hanging chats um yes Luca as you’re getting in and building um polls what do you think polls can tell us and what maybe not I think that

11:24 what polling does uh uh provide is uh a broader snapshot of electorate um and it can give you a general sense of uh What uh the electorate is thinking and uh but it’s not 100 accurate which I think has been a a problem with the way that people interpret polls uh especially in

11:45 the last couple of election Cycles because in a lot of these close races you know a point or two or three makes a big difference in the outcome uh but people hold pulling to such a high standard that uh the way that an election comes out they they expect polling to be down to the dot but the the problem is it’s just really tough to get it to that point of accuracy so I

12:07 think polling is a better uh measure uh for a less accurate uh not a 100 accurate read of the electorate but it gives you a good idea of what people are thinking you’re never going to get a perfect result with Poland yeah I I think that first for starters I think we as a profession have misled uh the

12:28 public in terms of expectations I’ve played that game for years and I came out almost always on the right side of close races um in terms of our record of accuracy but I always knew that that was ephemeral 10 to 14 of Voters now tell us an exit and post-election polls I made

12:51 up my mind on election which means that uh there’s a lot of fluidity a whole lot more food than you think 2016 I heard Luca just say that the polls were terrible when you look closely at the national and then the Statewide polls and Battleground States you will

13:12 it is unmistakable there was a trend line away from Hillary and towards Donald Trump that’s not because uh necessarily that uh on the day before the election they came out and said Hillary Clinton’s going to lose Pennsylvania or Hillary Clinton’s going to lose Hampshire Wisconsin it was

13:34 because each of the last nine days Hillary was going down a point or two and Trump was gaining that that trend line was unmistakable so what you’re saying that the trend was important that was the information value there is that you you could see there was a pretty strong signal it wasn’t it wasn’t vacillating all over the place no it was not vacillating all over the

13:56 place there were outliers I mean there were polls that didn’t capture it but for the most part if you know what you’re looking for and that’s the thing that I try to do when I do forums like this don’t rely on upholsterer who has an R after their name or a d after their name there’s going to be some spin involved look for good independent bowls

14:18 uh with tracker let’s talk a little bit about um the the practice of polling and whether or not that’s changing or evolving at all and kind of the electorates relationship to pollsters and John I’ll start with you and then we’re we’re going to come to Lucas as maybe a an interesting window into what could be coming but how how has the

14:40 relationship between the electorate and pollsters changed in the time you’ve been at this and you could talk about that and how people interact with folks on the phone who are calling to do surveys what do you see in that relationship over time when I started in late 1984 Stan averaged 65 response rates everybody had

15:02 a landline 96 households and culturally socially we answer the phone it was not uncommon to hear on the other end please be quiet somebody’s calling me from Newport and has questions to ask I’ve got to do that that was then

15:23 there were virtually no answering machine in 1984. uh and socially people were glad to answer the phone in fact here’s some that she wanted it wasn’t uncommon for an older woman to answer the phone say well no you don’t want to talk

15:45 that’s the demure housewife that does not exist anymore and it’s also a woman telling a total stranger hey I’m home all alone so yes things have changed dramatically with technology answering machines star six nine so on in our instance much more

16:06 reliant on online and also we do live phone calls and we do all cell phones cell phone is just not uh friendly uh technology Luca how did you conduct your polls and after you’re done answering that question I want to talk about maybe what you see in among your classmates

16:29 and peers and how they react to polls well let’s start with how’d you do yours how did you do your pulse uh we we didn’t have that much money to start off with so we couldn’t do uh you know one of the big operations that like ABC has with with real human beings calling all all these phone numbers so what we found is a system called ivr which is pretty

16:49 much a robot that calls a ton of phone numbers most of them phone numbers either don’t pick up or they hang up as soon as they hear that it’s a robot but if you do it enough times you’re going to get a suitable polling base so we we would just uh it’s 15 cents per call so uh we would be uh we were able to get away with paying you know two three thousand

17:09 dollars per poll where you know a a much larger uh sort of uh traditional polling operation would would have cost a lot more than that so uh what but what we had to do since uh we only had access to landlines um because of uh it’s some Federal Regulation that you can’t uh use uh uh

17:30 an automatic caller to call uh cell phone numbers so we could only uh use landlines uh we had to there was a lot of uh weighing involved with our polls because we weren’t getting a a very accurate sample of an electorate I mean because it’s much older uh much uh the people who have landlines now or don’t reflect the general makeup of the

17:52 electric so we had to uh you know get uh weigh our polls uh the the the once we weighed our polls we had a much different outcome than the world result what do you see in your peers your your friends your classmates and so on what what’s their relationship to polls do they even think about them

18:13 pay attention to them how do they how do they interact with polling they they don’t uh I don’t even think they think about it it’s people people do think a bit about politics but it it never gets to the Electoral level I mean people I I have never heard any of my classmates talking about polling or or how they think an election is going to turn out or

18:33 anything like that Luca how do you think Poland contributes to or supports self-government I think that polling uh it gives elected officials an insight into what people think I mean because otherwise you have to wait until an election to know what people are thinking but with polling uh politicians I mean they wanted most of

18:55 them want to do something that’s popular so they a lot of times if if polling shows that something is unpopular uh you know our elected leaders won’t do that and they’ll do what people support so I think it gives our leaders a really good insight into what people are thinking and it allows uh their views to be implemented without an election having

19:15 to take place or at least some of their views we all uh want to feel connected and want to know that you know in our inner cells we may have feelings but where do we stand with everybody else I think especially in a a modern era where there are so many ways to isolate it atomize but pulse do is it offers us a

19:36 connection uh to the rest of the world in a scientific way in terms of matters of public policy it’s not only in policy makers to understand where majorities are or consensus ideas are but it’s also that we measure intensity on a typical

19:57 issue you you may have 60 percent who agree and 40 who disagree but among that 40 who disagree there is a solid half of them but we’re so intense they’re ready to march in the streets with pots and pans make a lot of noise those are the sorts of things that that

20:20 polls have to try to measure that’s what moves policy makers but what Luca do you hope people will be watching for and how do you hope they will use polling and how you hope they will consume that data going into the next election cycle I think uh you know polling should not be as important as what they think is

20:41 best for the country I think people should uh choose their base their votes on on the candidate that they support instead of what they think polling indicates I I think what pulling highlights for this next election is that you know uh most people really want an alternative uh to what we have now uh people conduct polls and it’s like 70 80

21:01 percent want somebody new uh to to run in the election but as you can see that that’s probably not going to happen uh we’ve got the two uh Front Runners uh are very unpopular I think that what polling should do is offer an Insight uh offer people an insight into the way that the country is thinking because it

21:21 seems often that people uh they don’t understand that their Viewpoint is uh there are a lot of people with other views across the country so I think think that what polling can do is really illuminate to people that there are opposing views and and uh there’s a lot more diversity of thought than most

21:41 people realize in this country and I think that could be very useful I would ask that those who are on air or on the opinion Pages be smarter instead of a very one-dimensional reading and interpretation I wish that instead of looking always at the more apocalyptic

22:02 uh results of a poll how we disagree what separates us what makes us angry that uh those who interpret would take a look at a plenty of areas of defenses plenty of areas where there is potential bridges that be part of the conversation as well

22:22 and then the third thing I would say is I just wish we could get predictive and prediction out of the language you don’t forgive you suggest say who’s ahead say what a trend line is but then um the results come in on Election Day and

22:44 um not the pollsters fault or it’s not anybody’s fault all right gentlemen thank you very much for making time to speak with our audience today we appreciate what both of you are doing to help our fellow citizens of all of us make better decisions understand how the country is thinking about issues and hopefully that helps

23:05 motivate each of us to do more learning ourselves and make wiser choices thanks very much to both of you thank you good luck Luca thank you great meeting you