Is the blue wave forecast by polling real?

Prediction markets say it is, but they don’t seem to add much to polling.

A voter checks the prediction markets for the NY-12 Democratic primary. (Photo by Laura Brett/Getty Images)

Polls indicate that Democrats are in line for a very good midterm. They’re heavy favorites to win the House, poised to do well in gubernatorial races, and stand a good chance of flipping the Senate despite unfavorable geography (unless John Fetterman flips it back).

But is this right, or are the polls just massively oversampling Democrats? One way to check is to look at the prediction markets, which are maybe a little less bullish on Democrats’ fortunes than Nate Silver’s model but still say that Democrats are more likely to win a Senate seat in Texas or Alaska than Republicans are to win in Michigan.

The markets, in short, think the polls are basically right.

As a journalist, I do really like having prediction markets for this sort of gut check, although this appreciation exists awkwardly alongside my concerns about the social impact of widespread legalized sports gambling. (And to that end, perhaps I should clarify now that this post is most certainly not sponsored by anyone.)

But one reason I find it useful is that opinion writers are constantly trying to position ourselves relative to the conventional wisdom or what “people” think. But even with polling, it’s traditionally been hard to know what the conventional wisdom is. It’s useful to have a line on a chart that says very clearly there was a Marco Rubio boomlet earlier in the year but everyone is back to thinking the 2028 G.O.P. presidential nominee will be JD Vance.

Beyond acting as a sort of conventional wisdom index, though, the more interesting question is whether these markets provide additional insight into the underlying truth.

After all, a big problem with journalism is that while conveying accurate information is a big part of what journalism is for, the actual incentives for correctness and truth-telling are weak. One reason a lot of the best, most sober-minded reporting comes in business sections and in places like Bloomberg and the Financial Times is that’s where the value proposition of journalism comes closest to “the customer needs the information to be accurate.”

Similarly, when you’re betting, you want to be correct. The gamblers may be wrong (indeed, obviously a bunch of them are going to be wrong) but they’re at least on average genuinely trying to be correct. And not only could a truth engine be useful, but (per the point about the business press) to the extent that world affairs becomes an investable proposition, that creates a market for accurate information.

At the same time, these things are not oracular. Right before polls closed in Wisconsin, the gamblers had Francesca Hong not just as a favorite (as the polls said) but as an overwhelming favorite. There was no hint in the market data of what was to come.

Still, this is unfair in isolation. Things that a forecaster says have 97 percent odds of happening shouldn’t happen 100 percent of the time. They should happen 97 percent of the time. The right question is about calibration: Do things happen roughly as often as the markets say they should happen?

The answer, as far as I can tell, is “basically yes.” But at least with regard to American election outcomes this turns out to be hard to distinguish from “assuming the polls are correct is a good betting strategy.” We all know polling isn’t perfect, but markets seem to confirm that it’s decent.

What they don’t seem to do is give us a lot of extra information.

The primary-election markets were pretty well calibrated

Kalshi released its own study in August looking across all its markets and concluded that Kalshi markets are extremely well calibrated.

The dashed diagonal line represents a perfectly calibrated market. For example, events given a 60 percent probability would occur 60 percent of the time.

Of course, I don’t take Kalshi’s word for it. And beyond their calibration, I’m also interested in the question of whether these betting markets told us anything useful about elections. Kalshi and Polymarket both make their data available via A.P.I., and thanks to the miracle of modern artificial intelligence, even I am now capable of generating the code needed to pull it all and get the data for the primary-election markets.

When aggregating across both Kalshi and Polymarket and looking at every primary for which there was a market, you can see that the calibration is in fact pretty good. The main exception, predictable from our general knowledge of human psychology, is a small bias toward long shots.

But generally speaking, when the predictors bet that a candidate has a 75 percent chance of winning, that candidate wins about 75 percent of the time.

That’s not nothing. Indeed, this is precisely the nuance that written commentary often misses. Seventy-five percent is not a coin toss. It’s not neck-and-neck. If you think Hillary Clinton has a 75 percent chance of winning, it makes sense to focus most of your planning on preparing to cover a Hillary Clinton transition. But then if Donald Trump wins, that doesn’t mean you were “wrong” per se — the coin should come up heads twice in a row one time in any four. So this is decent performance.

Still, we’re left with the question of whether this really teaches us anything.

Philip Tetlock is one of the great prophets of prediction markets, and one of his core findings in both “Expert Political Judgment” and “Superforecasting” is that subject-matter experts tend to be surprisingly bad at predicting things within their area of expertise.

The people who are best at predictions, the superforecasters, tend to have a domain-agnostic skill.

It’s not a superpower. It’s a discipline they deliberately hone over time — making precise, quantified predictions and then looking back at their calibration and trying to improve. The promise of a prediction market in sports isn’t just that it can tell you the Spurs will do better than the Wizards (everyone knows that) but that it can harness forecasting skill to deliver better results than blowhard sports pundits.

For elections, fortunately, we don’t need to rely only on blowhard pundits. We have public opinion surveys — polls — that tell us who’s going to win the election.

Prediction markets mostly follow the polls

This is where it seems to me the markets are taking the L.

Predicting elections when you have access to good polling data just isn’t that hard. The candidate who’s up is usually going to win. If there’s a small polling lead, you should give the leader a small edge in the odds. If there’s a big polling lead, you should be very confident. It’s true this will occasionally lead you astray (like in Wisconsin), but there’s no great shame in that — it’s about calibration.

The problem is that it’s hard to find examples of the betting markets doing much more than processing the polling data as it comes in. What you get instead is a lot of cases of markets shifting strongly after surprising new poll results and then being vindicated.

But take a case like the Republican primary for Michigan’s Eighth Congressional District.

I didn’t think much about this race until it happened. It didn’t attract much coverage, and I didn’t see any public polling about it. Trump endorsed a guy named Amir Hassan, and the markets assumed Hassan would win, but he ended up losing to Thomas J. Smith, whose name was on the ballot even though he’d dropped out.

“The guy endorsed by Trump will defeat the guy who already dropped out” was a perfectly reasonable assumption. Given that it turned out the other way, though, it’s pretty obvious what happened — Republican primary voters like the name “Thomas Smith” a lot more than the name “Amir Hassan,” and precisely because nobody was paying attention to this race, they probably didn’t realize that Trump was backing Hassan and Smith had dropped out. The betting odds do not display any foreknowledge of this whatsoever, even though it’s also the kind of thing where you might expect a shrewd generalist could have added value.

On the other hand, there’s only $5,367 of total volume in the market.

There are a bunch more races like this — TN-4, TN-6, MO-4, MO-8 — where inconsequential primaries received minimal coverage and no public polling and where the ultimate winner was a huge underdog in thinly traded betting markets. It’s not that every unpolled race is like this, of course. In fact, most unpolled races just featured an overwhelming frontrunner (an incumbent facing a token primary challenge, say) who was heavily favored in prediction markets the whole time and then won.

Betting on politics isn’t very lucrative

The point is that it’s hard to find a marquee example of a race where the private information aggregated in the market was really onto something.

The markets are a good index of conventional wisdom, which is mostly heuristics like “the candidate up in the polls will win” or “in the absence of polling, assume the incumbent and/or the candidate Trump endorsed will win.” And what we see in the calibration is that those heuristics are pretty good!

But sometimes you get a race like the Democratic primary in CO-3 that attracted very little national interest, polling, or coverage and where there was no obvious reason to see one candidate or the other as the frontrunner. Markets narrowly favored Alex Kelloff, but actually Dwayne Romero won.

In retrospect, Romero’s victory can be seen as a data point in favor of one of my pet theories: that white candidates whose names end in vowels overperform with Hispanic voters (see James Talarico, Jim Costa, John Duarte, David Valadao for other examples). Maybe I should have seen Romero selling at 36.5 a month before the election and bought in on the basis of vowel theory.

Given what we know, though, the real way you’d expect markets to generate information about a race like this would be by getting people to commission polls. We know that betting markets mostly follow polling and that polling is mostly pretty accurate. So if you commissioned a good poll of this race, you’d probably have seen that the markets were underrating Romero and had a chance to buy. But the total trading volume on this race was tiny — $72,000 or so on Kalshi — so it doesn’t really seem like this would be a worthwhile investment.

Another potential source of information is private polling. It’s possible some of the super PACs or party committees that were polling Colorado to get information about other races in the state actually had CO-3 data. People with access to that information could trade on their private knowledge, but that violates the markets’ insider trading rules and again doesn’t seem like something that would be lucrative enough to be worth losing your job over.

And that seems to me like the limit of these markets at this point.

There’s enough publicly available information and skilled traders to make election markets well calibrated. And theoretically the markets could generate even more useful information by spurring people to invest in high-quality polling. After all, one huge problem with public polling is that if you tell a media outlet you could make their horse-race poll slightly more accurate by tripling the cost, it’s not really clear why you’d spend the money on that. A gambler seeking a private edge might, and his bets could convey new information to the public.

But the market for random House primaries isn’t like the market for global oil futures or a major stock exchange where gaining a small edge can be quite lucrative. So we’re left, for now at least, with what seems to me like an efficient processor of public information and a good index of conventional wisdom but not necessarily a source of fresh insights.