It makes me wonder whether Cook, Sabato et al. May be closer to the mark than the polling. G. Elliott Morris just had a piece up that indicated that the prediction models tended to give an advantage to the incumbent president’s party in mid-term elections relative to the polls 45 days out.
I think Cook & co are also just a bit slow and put everything in a toss-up. They weren't any more accurate than models in the past.
And Elliot is right, they do always underestimate the out-of-power party in midterms, so that's good for Democrats. I think it mostly has to do with Cook & Co overstating the results of the presidential election 2 years ago, thinking incumbents can hold on against a blue wave environment with a Generic Ballot of D+8
This is an interesting question and design. I like the tables. I apologize if I am just missing this, but I'm not able to unpack this paragraph, which is summarizing some important findings:
"Key races saw countless election polls that heavily influenced 538’s forecast, while obscure contests were largely built on fundamentals. While the largest misses happened in these races, they were more random and not systemic. Races that were heavily polled had smaller misses, but all favored Democrats. My read: Late, Republican-associated polls (by the likes of Insider Advantage & Trafalgar) overstated Republican support and tilted the averages in their favor."
In the third sentence, I think you mean heavily polled races in 2018 had small pro-Dem misses, notraces in 2018 and 2022 - is that right? Also, in the last sentence, which polling averages were tilted in favor of Rs by late, Republican-associated polls? Is that a 2022-only effect, or is there a more complicated interaction between other polls and R-affiliated polls?
You raise a good point! Sorry, I could have been more precise with this section.
In 2018, the average 538 forecast miss was the same for key races & all other races. In 2022, however, there was a big difference: Democrats were underestimated by 3 pts in key races, but across all senate races, there was basically no forecast miss.
My idea: The only real difference between key races and all other races is that key races are getting polled A LOT by high quality pollsters, while obscure races do not. So, key races have a lot of polls in their averages, and the polling became a bigger part in a 538's prediction. Therefore polling might have been the reason why 538's 2022 miss was isolated to key races.
About the last sentence: A lot of these polling averages shifted heavily to the right ahead of Election day, after Republican aligned pollsters dumped poll after poll. For example: 3 of the 5 polls in Real Clear Politics' final 2022 Pennsylvania Senate average came from Republican aligned pollsters, who tilted the average in OZ's favor.
I don't know if that phenomenon will repeat itself, but sometimes it's best for polling averages to tune out pollsters like Rasmussen or Trafalger.
Thank you! There's a lot to gain from looking closer at polls. I'm currently preparing an article about undecided voters in recent polls for today or tomorrow :)
Great analysis! I think you're correct about 2022, which wasn't a bad miss for the polling industry as a whole like 2020. I calculated that the average pollster in Senate races was about 2% more Republican leaning than pollsters that surveyed the generic ballot (probably representative of the broader industry). Hopefully quality pollsters on both sides of the aisle won't shy away from state polling this year.
I feel like a lot of pollsters have changed their methodology around 2024 to avoid underestimating Trump again. I don't really remember recall-vote being a thing in most polls before that (I believe the NYT once even wrote about why it can be misleading).
Still, I am glad about any quality state poll that isn't another Dem internal
A lot of the primary polls in 2026 seemed to have a new issue: they overestimated DSA support (thinking about Michigan & Wisconsin). Do you anticipate that being an issue in the general for more progressive candidates like El-Sayed?
It makes me wonder whether Cook, Sabato et al. May be closer to the mark than the polling. G. Elliott Morris just had a piece up that indicated that the prediction models tended to give an advantage to the incumbent president’s party in mid-term elections relative to the polls 45 days out.
I think Cook & co are also just a bit slow and put everything in a toss-up. They weren't any more accurate than models in the past.
And Elliot is right, they do always underestimate the out-of-power party in midterms, so that's good for Democrats. I think it mostly has to do with Cook & Co overstating the results of the presidential election 2 years ago, thinking incumbents can hold on against a blue wave environment with a Generic Ballot of D+8
This is an interesting question and design. I like the tables. I apologize if I am just missing this, but I'm not able to unpack this paragraph, which is summarizing some important findings:
"Key races saw countless election polls that heavily influenced 538’s forecast, while obscure contests were largely built on fundamentals. While the largest misses happened in these races, they were more random and not systemic. Races that were heavily polled had smaller misses, but all favored Democrats. My read: Late, Republican-associated polls (by the likes of Insider Advantage & Trafalgar) overstated Republican support and tilted the averages in their favor."
In the third sentence, I think you mean heavily polled races in 2018 had small pro-Dem misses, notraces in 2018 and 2022 - is that right? Also, in the last sentence, which polling averages were tilted in favor of Rs by late, Republican-associated polls? Is that a 2022-only effect, or is there a more complicated interaction between other polls and R-affiliated polls?
You raise a good point! Sorry, I could have been more precise with this section.
In 2018, the average 538 forecast miss was the same for key races & all other races. In 2022, however, there was a big difference: Democrats were underestimated by 3 pts in key races, but across all senate races, there was basically no forecast miss.
My idea: The only real difference between key races and all other races is that key races are getting polled A LOT by high quality pollsters, while obscure races do not. So, key races have a lot of polls in their averages, and the polling became a bigger part in a 538's prediction. Therefore polling might have been the reason why 538's 2022 miss was isolated to key races.
About the last sentence: A lot of these polling averages shifted heavily to the right ahead of Election day, after Republican aligned pollsters dumped poll after poll. For example: 3 of the 5 polls in Real Clear Politics' final 2022 Pennsylvania Senate average came from Republican aligned pollsters, who tilted the average in OZ's favor.
I don't know if that phenomenon will repeat itself, but sometimes it's best for polling averages to tune out pollsters like Rasmussen or Trafalger.
Thanks - that's a subtle and interesting observation.
Thank you! There's a lot to gain from looking closer at polls. I'm currently preparing an article about undecided voters in recent polls for today or tomorrow :)
Great analysis! I think you're correct about 2022, which wasn't a bad miss for the polling industry as a whole like 2020. I calculated that the average pollster in Senate races was about 2% more Republican leaning than pollsters that surveyed the generic ballot (probably representative of the broader industry). Hopefully quality pollsters on both sides of the aisle won't shy away from state polling this year.
I feel like a lot of pollsters have changed their methodology around 2024 to avoid underestimating Trump again. I don't really remember recall-vote being a thing in most polls before that (I believe the NYT once even wrote about why it can be misleading).
Still, I am glad about any quality state poll that isn't another Dem internal
A lot of the primary polls in 2026 seemed to have a new issue: they overestimated DSA support (thinking about Michigan & Wisconsin). Do you anticipate that being an issue in the general for more progressive candidates like El-Sayed?