RealCarlAllen & RealRickRosner on Real Polls

Author(s): Scott Douglas Jacobsen

Publication (Outlet/Website): Personal SubStack

Publication Date (yyyy/mm/dd): 2024/11/01

Rick Rosner: So, I feel that in this election, there aren’t a lot of clear signals, but I found one indicator that consistently provides clear and accurate results about voter preferences, and that’s counting the votes.

Carl Allen: Yes, that’s the best way to do it. Yes, that’s the best way to do it. Not everyone wants to do that, though. Some people would rather not count all the votes, but that is the best way to determine who wins.

Rosner: Aside from that, are there any clear indicators? Exit polls make me a little uneasy because they serve other purposes, but mostly they just fill airtime so that news outlets have something to discuss during the 12 hours between the polls opening and closing.

Allen: In other countries, exit polls are much more accurate because they know who has voted. They’re legitimate exit polls, and most people vote at the polling place itself. In the U.S., however, there’s a significant discrepancy due to vote-by-mail, early voting, and in-person voting on Election Day. This makes it harder to aggregate the numbers because every poll has a margin of error, and in these cases, you end up stacking multiple margins of error on top of each other.

So, exit polls are useful — more useful than no exit polls — but when we’re looking at close elections across swing states, if I see a 52–48 split either way, I don’t take that as a definitive victory for any candidate. It’s simply too close, especially when we’re analyzing exit poll data.

Rosner: Are there clear indicators this time around? There are a few things I look at to give myself hope that my preferred candidate will win. One indicator is that it appears women are out-voting men 55 to 45. However, that’s an uncertain statistic because not all states release that demographic data consistently. But, you can see gender numbers across all states on news sites like NBC. I don’t know where they’re getting that data, and sometimes it feels like they’re making it up.

Allen: However, they’re not; there’s always solid data. Different states provide different demographic breakdowns for their polls. For example, Pennsylvania and Nevada provide a detailed breakdown of party affiliation, age, gender, and more, while other states, like Michigan and Georgia, only provide limited demographic information such as age, race, and a few other data points.

For Harris, it’s encouraging that women are voting more early, but this has always been the case. Historically, women have consistently outperformed men in early voting, and Democrats have generally outperformed Republicans in this regard. However, the gap has narrowed between Democrats and Republicans.

Republicans are now encouraging their supporters to vote early more than ever before. Still, there’s very limited data, and this is how I explain it to people: the polls indicate a close race. Harris is favored because she’s leading in the Blue Wall states, while other key states are toss-ups. But the polls continue to indicate a close race, and all the early voting data we’re obsessing over, no matter how it’s analyzed, still suggests a close race.

For example, Pennsylvania looks better for Democrats than Nevada, but Nevada has always been an anomaly with its early voting data. It’s always been a bit of an outlier. As statisticians and analysts, we know that states are all correlated in some ways, but Nevada remains unpredictable.

So what Pennsylvania does isn’t entirely independent of what Michigan does because people tend to act similarly. But Nevada — they’ve always been a bit of an outlier. So, all we know from the early vote data is that neither candidate will win in a blowout, but we already pretty much knew that, right?

Rosner: I have a show where I argue with a Trump-supporting guy, and we were doing that last night. Atlas Intel came up — they’re a pollster. Yep. He, of course, you’ve trained me pretty well by now. He said, “Atlas Intel seems to be one of the better pollsters,” and I was like, no.

They were the most accurate in 2020, and I responded, “Yes, but that accuracy doesn’t mean much.”

Allen: Oh my god, yes. So, this pattern that people have been brainwashed into believing — and I use the word brainwashed intentionally here because it’s teaching people to believe something that’s not true — is that whichever pollster is closest to the final election margin is therefore the most accurate. It incorporates all sorts of unscientific assumptions. One example I use is, if I roll two dice, we know that the average over a long period of time is going to be 7. The most common outcome of an individual roll will be 7. But if I say, “No, it’s going to be 5,” and you say, “I’m going to pick 7 because 7 is the most likely outcome,” well, there’s about a 1 in 3 chance that I could be closer than you — just by luck. For random, literal chance.

So we have to be very careful when we’re talking about a sample size of 1, which is the 2020 presidential election, and saying, “Oh, they were closest on the margin.” Even if I granted that this is a legitimate metric for measuring poll accuracy — which it’s not — it’s not actually measuring poll accuracy. But even if I grant it, there’s a term in legal speak where we grant, or use, the logic of argument —

Rosner: Yes, for the sake of argument.

Allen: Even if I were to grant that, it still doesn’t prove anything. It just means they were directionally closer than other pollsters. When you look at 2020, 2018, 2016 — the math — if you do the math on how many elections we would need to truly figure out who the most accurate pollsters are, we’re talking hundreds of years. Because an election happens only every four years, so we’d need sample sizes in the dozens, if not hundreds, to quantify who the most accurate pollsters are. And that’s obviously not realistic, assuming their methodologies never change.

Scott Douglas Jacobsen: Yeah, and it also assumes that those pollsters live that long.

Allen: Yes. There are too many faulty assumptions to say a pollster was the most accurate just because they were closest in one instance, and that’s the unscientific mindset that Silver and the mainstream analysts and experts have brainwashed people into believing. They don’t just accept this, they actively promote it themselves, which demonstrates that they do not uphold any scientific standards.

Rosner: So, my question for you is: Do sports playoffs and sportscasting make you crazy because they have no way of determining who the best team is, since they don’t play enough games?

Allen: That is absolutely correct. The thing about sample size in playoffs is that you have a 3-game, 5-game, or 7-game series — sometimes just one game — but the team that wins advances. Statistically speaking, the best team is less likely to win the championship than a lower-ranked team.

Because you’re dealing with, in any given season, maybe 5 to 10 teams that are good enough to win the championship. Statistically, the best team might only have a 10%, 20%, or 30% chance of winning. So, the best team doesn’t regularly win, which messes with our perception because it doesn’t seem to make sense.

People think, “Well, they’re the best team, so shouldn’t they win?” That’s when I go back to the dice analogy. I say, “The most common roll is a 7, but the most common outcome is still less likely than all the other outcomes combined.”

Rosner: Yes. I was watching the World Series, and baseball is perhaps one of the most random major sports. Yet, they constantly explain the pitcher’s intentions and what they were trying to do with the batter. They act like every outcome is planned, and it’s like, no. It’s maybe 20% what the pitcher wanted and 80% randomness.

Allen: That’s the uncomfortable truth about human performance. One of my first areas of research was athletic performance — how to maximize it, improve it, and find an edge. People have a hard time believing that we can quantify human performance because there’s so much randomness involved. It’s really hard. So, when we’re talking about athletes at the elite level, everything can be expressed as a probability.

Take Anthony Volpe hitting a home run yesterday. Volpe didn’t hit many home runs in the regular season, so it was highly unlikely that he would hit one, and yet, he did. Unlikely events happen all the time, but in sports, we have a huge sample size — a 162-game regular season, with about 4 at-bats per game, give or take.

So we have this huge sample size in sports where, when rare things happen, we kind of accept it as, “Wow, that was great, that was crazy.” But when we’re talking about politics, where our sample size is an election every 2 to 4 years, when rare things happen, people perceive it as, “Someone did the math wrong.” So, when Anthony Volpe stepped to the plate, I would have said he had probably a 1-in-30 or 1-in-40 chance of hitting a home run.

And he did. Was your math wrong? Maybe, maybe not, but rare events happen all the time. That’s hard to process when we’re thinking about deterministic outcomes, like winning and losing an election.

Rosner: Speaking of tricky math, what’s happening this year is similar to what happened in 2022, and maybe a bit in 2020. You have pollsters who are trying to be accurate, and then you have pollsters with an agenda. They might still be trying to be accurate but are also pushing certain insights.

I don’t know how hard they’re trying to be accurate, but I do know that if you go to their home page, you’ll see they hate Kamala Harris. There are 8 articles about how she and the Democrats are the worst people on earth, and here’s how Trump can beat them. And then they release a daily poll on Harris versus Trump nationally.

And because it’s a daily poll, it overshadows all the other polls, which are usually weekly. So you look at the list of recent polls, and 5 out of 8 of them are from Tip Insights.

Allen: Yes. Their methodology seems fine as far as I can tell, but it’s interesting to see pollsters with a clear bias. You have to figure out how that bias might influence their work. In my book, I wrote about this at length because it’s important to note that everyone has biases. Most pollsters are biased because you don’t get into polling without an interest in politics, and you don’t have an interest in politics without having some political bias. So, the fact that pollsters themselves can be biased doesn’t mean their data isn’t reliable or valid.

That’s why we need transparent standards — like who they’re contacting and how they’re weighting the data. Nate Silver’s operating hypothesis is that pollsters will always act with accuracy in mind. He believes that the pressure from within the field will push them to act with accuracy because that’s how they’re ultimately judged. But starting before 2022, and especially as I was finishing research for my book in 2022, I noticed an interesting phenomenon: pollsters who were releasing boring polls — showing close races or whatever, without a big headline — weren’t getting much attention.

But these partisan pollsters were putting out headlines like “Oz is ahead” or “Lake is leading” or some underdog candidate was winning, and those polls blew up. They got tons of attention. So I started thinking: the reason pollsters release their poll data in the first place is to promote their brand or their work. Political polling is a small part of what these population research companies do. The political polling is the thing they almost give away for free — or very cheaply. They break even or take a loss on it to promote their other work elsewhere.

Rosner: Why don’t you do political consulting?

Allen: So I said, wait a minute. If they’re doing this work almost for free to promote what they’re selling, they have a clear idea of who their target market is.

Right, it’s easy for me to calculate that some of these partisan actors might not be acting with accuracy in mind. If releasing a poll that says, “49–46, Trump is ahead” makes them more money — when their actual data might show 47–47 or 48–48 — that’s an easy choice for them, especially when the race is that close. So, we have to be careful. I understand that pollsters aren’t always prioritizing accuracy, and the belief that all pollsters have the same intentions is a little misplaced and, frankly, dangerous.

Rosner: So, I noticed you’re wearing a wedding ring. When I try to talk to my wife about, say, physics, she’ll go, “Oh my god, talk to me about something that isn’t boring.” When you talk to your wife about the intricacies of polling, does she have a time limit before she says, “Okay, enough”?

Allen: My wife is extremely intelligent. My intelligence is more laser-focused, so when I want to learn something, I go very deep into it. She’s incredibly smart in that she picks up things quickly. So, when I talk to her about poll data, she’s engaged and asks, “What about this?” And that’s usually the next thing I’ve been researching. She’s on the same page as me, but she has a lot of anxiety around political outcomes because she’s a normal human being who cares about the state of the world.

The idea that so many people could vote for someone she finds reprehensible really makes her upset. So, we don’t talk too much about the political side of things. But when we focus on the statistics behind polls, she’s a great resource for me because she’s so intelligent, but she hasn’t studied polling data intensely. So, I explain things to her, and if she doesn’t understand it right away, that means I’m not explaining it well enough. That’s how I improve my writing and speaking — I know that if I’m explaining something clearly and someone doesn’t get it, there’s a gap that I need to address.

With her, if she doesn’t grasp it immediately, I know I need to adjust my explanation. That’s helped me get better at explaining things over time.

Rosner: Yes, my wife’s smart too, but her intelligence is more focused in a literary direction. She can predict what’s going to happen next on any TV show we’re watching. Yes, we have a game where we guess the next line or the next scene.

Allen: Right, I’ve talked to my wife for hours — literal hours — about my book and research. She finds it interesting, and she gives me feedback, which is helpful. She’s definitely a resource for me — the biggest resource for writing my book, no question. She’s challenged me on things, given me ideas, suggested ways to explain concepts, pointed out good and bad analogies. And that’s been invaluable because most people don’t want to help me write the book.

People want the book itself, they want the outcome. They don’t see the 1,000 pages I’ve written that had to be edited down to 300, so it flows and makes sense. She’s helped me through that process.

Rosner: So, with the election six days away, there must be a certain amount of stress in your house. My wife’s been asking, “What’s going to happen?” I’ve had to cut down on coffee because, over the past couple of weeks, I sit at my computer in the morning, and suddenly my heart is racing at 110 beats per minute as I read all the rancorous tweets and the news.

Allen: We all deal with anxiety differently. I’ve always dealt with it through research and study. Weirdly, I’m not that nervous — not because I know what’s going to happen. I do think Kamala Harris is in a good position to win, which helps me feel a bit better. But I’m aware that Trump could win. My lack of anxiety comes from accepting that it will be close, the outcome is uncertain, and there’s only so much I can do to influence it. That’s the only way I’ve been able to detach myself from the anxiety of it all. This election will probably shape not just the next four years but the next 40. The only way I’ve been able to detach myself from that reality is to look at everything as a range of outcomes and probabilities, and then do what I can to influence the outcome favorably. But, ultimately, there’s only so much we can do. So, we have to let ourselves sleep at night.

Allen: Let ourselves sleep at night. So, go ahead.

Jacobsen: There are more women voting than ever before. In your conversations with your wives, has there been any speculation or data that speaks to the gendered aspects of voting trends? For example, reproductive rights being important to many women — does that become almost the sole decider for them?

Allen: The early voting numbers definitely show a lot of women voting. But this isn’t out of the ordinary, so we have to be careful about inferring too much from it. That said, we know women are angry. More women are favoring Democrats than in previous elections. I didn’t look as far back as Obama because he won by such a large margin, but women are supporting Democrats more than in recent elections, and white women are favoring Democrats more than before.

There’s definitely something there — reproductive rights, decency, and maybe even the fact that Kamala Harris could be the first female president. I do think she’s getting more support from women because of that. Also, young voters, especially young women, are very concerned about the direction of the country. Young women, more than ever, are engaged.

If young women turn out in the numbers they could, this election wouldn’t be close. The problem is that young people, in general, aren’t as frequent voters. So, that’s definitely a cause for concern, but also maybe a cause for optimism for Democrats, especially across the swing states. The conversations I’ve had with the women in my life — my wife, my friends, my mother-in-law — it’s clear that the people I associate with share my political leanings more often than not. But there have been a lot of cases of women disowning men in their lives over political preferences. In years past, that might’ve been seen as extreme.

But as we’ve moved into this more polarized political climate — take Florida, for example, with the six-week abortion ban — it’s so extreme that women who are past that mark and have a miscarriage are often not allowed to terminate the pregnancy, putting their lives at risk. As men, we need to understand this is basic health care. This isn’t about birth control; this is essential health care for women. And here we are, three guys talking about this, so maybe we need to have some women on.

But we must protect these basic health care rights for women. This is showing up in the poll data — these extreme positions aren’t popular. I live in Ohio, and in 2023, we rejected an extreme abortion ban. States like Kansas and Kentucky did the same. Hopefully, Florida will overturn its six-week ban as well.

Allen: Yes.

Rosner: There’s a lot of work that still needs to be done, and it’s clear in the data that these extreme stances are unpopular. But in Florida, a measure needs 60% to pass, which is a very high threshold. Ohio’s measure also needed 60%, and it passed, but it’s a tough requirement for something that most people would agree is a basic health care right.

Rosner: There’s an ad out there with the message: “Your husband doesn’t know who you’re voting for.” It shows two couples at a polling place, and the men are glancing over, trying to supervise the women, but the women are ignoring them, looking at each other like, “We’re Republicans, but we’re secretly voting for Harris.”

Allen: Yes, I saw that. It’s powerful. There’s definitely intent behind those ads — not just encouraging women to vote independently of how their husbands are voting, but also making voting for Kamala Harris seem like a cool thing to do, almost like a secret act of rebellion. From an advertising standpoint, it’s powerful because it grabs your attention. But I do think it’s an effective way of reaching women, showing them that they can vote their conscience — that their husband or even their own political leanings don’t have to dictate their vote. You don’t always have to vote Republican just because you identify as one.

Allen:

Rosner: If young women turn out in the numbers they could, this election wouldn’t be close. The problem is that young people, in general, aren’t as frequent voters. So, that’s definitely a cause for concern, but also maybe a cause for optimism for Democrats, especially across the swing states. The conversations I’ve had with the women in my life — my wife, my friends, my mother-in-law — it’s clear that the people I associate with share my political leanings more often than not. But there have been a lot of cases of women disowning men in their lives over political preferences. In years past, that might’ve been seen as extreme.

Allen: Right.

Rosner: But as we’ve moved into this more polarized political climate — take Florida, for example, with the six-week abortion ban — it’s so extreme that women who are past that mark and have a miscarriage are often not allowed to terminate the pregnancy, putting their lives at risk. As men, we need to understand this is basic health care. This isn’t about birth control; this is essential health care for women. And here we are, three guys talking about this, so maybe we need to have some women on.

Allen: Absolutely.

Rosner: But we must protect these basic health care rights for women. This is showing up in the poll data — these extreme positions aren’t popular. I live in Ohio, and in 2023, we rejected an extreme abortion ban. States like Kansas and Kentucky did the same. Hopefully, Florida will overturn its six-week ban as well.

Allen: Yes.

Rosner: There’s a lot of work that still needs to be done, and it’s clear in the data that these extreme stances are unpopular. But in Florida, a measure needs 60% to pass, which is a very high threshold. Ohio’s measure also needed 60%, and it passed, but it’s a tough requirement for something that most people would agree is a basic health care right. There’s an ad out there with the message: “Your husband doesn’t know who you’re voting for.” It shows two couples at a polling place, and the men are glancing over, trying to supervise the women, but the women are ignoring them, looking at each other like, “We’re Republicans, but we’re secretly voting for Harris.”

Allen: Yes, I saw that. It’s powerful.

Rosner: There’s definitely intent behind those ads — not just encouraging women to vote independently of how their husbands are voting, but also making voting for Kamala Harris seem like a cool thing to do, almost like a secret act of rebellion. From an advertising standpoint, it’s powerful because it grabs your attention. But I do think it’s an effective way of reaching women, showing them that they can vote their conscience — that their husband or even their own political leanings don’t have to dictate their vote. You don’t always have to vote Republican just because you identify as one.

Allen: In this election, I think more than ever, we’re going to see, and this shows up in the data as A lot of traditionally Republican voters are going to cross over. You mentioned young voters, and yeah, they’re even less predictable than some other demographics because they tend to vote late. So the question is, will they vote later or not show up? That’s the hardest calculation to make.

When looking at poll data, we’re distinguishing between registered and likely voters. In the past, registered voters who didn’t vote tended to favour Democrats, but they didn’t show up. This year, we’re seeing that many Trump voters — or people who would favour Trump — are the less likely voters. It’s not in great numbers, but it’s showing up on some likely voter screens.

Yes. What’s playing out across the swing states, where we have substantial poll data, is that young people greatly favour the Democratic Party. But it’s not just young people; it’s young women. Young men also favour Kamala Harris, but not by as wide a margin. So, getting these voters to turn out is key. We’re seeing this in early voting data, where in 2020 and 2022, Democrats had huge leads. But in 2024, it’s smaller. I look at how many people have voted and how many are still likely to vote. You’d always rather have a vote banked than a vote that’s just likely.

What we’re seeing is kind of a regression to pre-COVID trends. Democrats are still voting, but fewer are voting by mail or early. The last weekend before Election Day is going to be critical. Democrats aren’t running up million-vote firewalls in Pennsylvania anymore because, number one, fewer are voting early, and number two, more Republicans are voting early. Those early Republican votes are skewing the numbers, which used to show Democrats with big leads and now have people worried.

Yes, but in reality, you can’t vote twice. So, the early vote numbers are just part of the story. If young people don’t turn out, Democrats might have a bad night, but the get-out-the-vote campaign is in full swing, and I think young voters are extremely motivated. We’ll have a better idea by Monday, as that last weekend is typically the highest for early voting.

Rosner: So, you’ve got a set of models in your head. When something happens — like a comedian at a Trump rally insults Puerto Ricans, and then three days later, Biden misspeaks and insults all MAGA supporters — does that adjust your mental model?

Allen: Not so much. Poll data will always take precedence over what I think. For example, Trump’s favorability ratings are higher than four years ago, and even though that doesn’t make sense to me, I put it in the model. I use that data to understand how undecideds might break and so on. However, with the Puerto Rican comment, each state has unique demographics. Florida, Pennsylvania, and a few other states have large Puerto Rican populations, and if those voters become more motivated, we know who they’re likely to support.

That comedian’s comment activated those communities, which are not detached from politics but may not have been as motivated until something like this happens. To their credit, the Harris campaign immediately capitalized on it and spread it widely. It was a terrible joke and a poor look for the Trump campaign. Now that these communities have been reactivated, Republicans in states like Pennsylvania might see worse numbers.

Yes, this could have a downstream effect, especially in places like Michigan and New York, where important House races are happening.

As for Biden’s comments, I don’t see them having as big of an effect. It was a bad line, and he probably misspoke, but it wasn’t Kamala Harris saying it or someone with pre-approved lines. It was an offhand comment that, even if it was as bad as it sounded…

License & Copyright

In-Sight Publishing by Scott Douglas Jacobsen is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. ©Scott Douglas Jacobsen and In-Sight Publishing 2012-Present. Unauthorized use or duplication of material without express permission from Scott Douglas Jacobsen strictly prohibited, excerpts and links must use full credit to Scott Douglas Jacobsen and In-Sight Publishing with direction to the original content.