The Prediction Markets Underestimated Polling Error
Twitter’s infinite scroll contains many triumphant posts from betting market evangelists. Venture capitalists invested in prediction markets are particularly euphoric: “mainstream media is dead”, “polling is dead” and “prediction markets are the future”.
The hypothesis that “the prediction markets won” has a pretty simple flaw: prediction market participants actually put tremendous faith in the polls. The pricing indicated that the polls were spot on and swing states would be extremely close. They also suggested Kamala was a prohibitive favorite to win the popular vote.
By contract, most polling-based models correctly suggested that you should bet against narrow Trump or Kamala wins in the swing states; correlated errors were more likely. Regarding the popular vote, there was more modeling disagreement.
The below table shows a sample of Kalshi and Polymarket pricing the morning of the election vs. probabilities generated by my models. My methods were extremely simple: I modeled the final margin of victory as normally distributed with a mean of the polling lead and a standard deviation that was between 3% and 5.5%,depending on the number of undecideds and historical polling error in that state. Wisconsin did end up with a margin of victory within 1%, but only by a hair. All other 49 states were not within 1%. My simple methods paid off handsomely.1 Other models like Nate Silver’s also suggested to go long polling error in these swing state margin of victory markets.2
To their credit, betting market participants identified that Biden was not guaranteed to run again, that JD Vance would be the VP nominee and that Trump was the favorite. But they messed up these margin of victory markets pretty badly, and related markets like “what will be the closest state” had even more bizarre pricing. If you see betting market triumphalists dunking on Nate Silver or pollsters… humble them by referencing these markets.
If you have a Kalshi account, you can track me as “parrot” here: https://kalshi.com/social/leaderboard. I also like betting on politics at physical casinos and Interactive Brokers, but they do not have public leaderboards.
Nate and many other modelers agreed with the betting markets on the popular vote. I think this was a tough modeling choice. The major national polls showed a tied race in the weeks before the election, implying a roughly 50% shot Trump would win the popular vote. But swing state polls combined with implied correlations between states implied a Harris +~2 national environment. The danger of this type of modeling was that the implied correlations would be off… as they were. Harris over-performed in swing states and underperformed the polls in non-swing states. The national polls were likely capturing this reality. In general, I think it is wise to be suspicious of models that rely heavily on covariance matrices generated from small numbers of events.
great post!