Calibration is not the only thing we need from prediction markets. Guessing either candidate with equal probability is perfectly calibrated.
What we also care about is Discrimination. Guessing 97% (and being well calibrated) is a lot more useful than guessing 50% (and being well calibrated.)
This is a great comment on how the prediction markets can validate poll results and even go beyond that but the markets are still very new in terms of public acceptance and awareness! The only way to make good money is to go against the grain and make a contrarian bet! But that often doesn’t work out because the bets are heavily based on evidence (ie polls) And I’m glad that you mentioned Tetlocks books, they are worth reading if you’re interested in prediction markets, especially Super Forecasters!
Personally I try to bet in markets where I think the polls might be wrong or the market is overrating something. I feel like for Dem Primaries the Party Decides is a framework that still holds up reasonably well so I made 700ish on my small bet against and lost a bit on the margin for the Michigan Primary but came pretty close there.
Prediction markets offer benefits only, and to the extent that, they can incentivize the disclosure of material non-public information to the market. The platform's fear of being soft on insider trading and the government's attempts to use standards from stock exchanges have turned them into nothing but fancy betting applications.
I mean, should you spend a bunch of money doing research to figure out who will really win so you can make money on Kalshi? Well, problem 1 is that their TOS technically ban anyone who has a formal influence over the outcome from participating and you are a voter. Problem 2 is the fuzziness of the lines on material non-public information. And hanging over all of this is the tension behind the SCOTUS ruling limiting insider trading to breach of fiduciary duty and theories of fraud against counterparties.
Yes, I know the lawyers will be able to draw some distinctions but the point is that for the behaviors that would actually make them useful -- eg incentivizing the production and sale of material information about the outcome -- have too much
legal risk to truly create the market or incentives.
Maybe we don't actually care about which google search term is the most common but imagine we did. The whole idea is that the prediction market should incentivize google to sell that data so we learn about it sooner. But even if that Google employee who bet on the most popular search terms had been contractually authorized to sample the raw search term data a month before it gets finalized and bet on it they'd still likely get prosecuted.
But how different is that from paying to run a voting survey? Where is the line? The fuzziness kills the very point.
1) I highly recommend Jesse Richardson's substack blog (Golden Path), as he's a 7-figure winning election bettor and has interesting things to say as a man who is fading the polling average.
2) In general, I think the real value of prediction markets in the U.S. tends to be when we have polls and early vote.
3) I mostly bet on sports, but I wrote this in direct contradiction to your point, as I think betting the polling average has actually never been less +ev than the present! https://substack.com/home/post/p-189914255.
It would seem to me that the focus of the news business ought to be on things that have actually happened, as opposed to predictions of what is yet to happen. (Or, for that matter, evaluations of those predictions.)
It is a very bad sign that so much anger is spilled over polls (or prediction markets) getting an election "wrong." With all the problems our country faces, all the signs of our decline, in effect journalists are re-focusing people on what is, at its core, entertainment. With all sorts of negative side effects:
1) Giving aid and comfort to those who would deny election results,
2) Adding to the general disillusionment with scientific inquiry as a means to know things
3) Legitimizing the slide from campaigns aiming to persuade the electorate to hyper-focusing on crucial groups or locations -- which, when you think about it, makes a mockery of our form of government
4) Inadvertently telling various viewpoints that they haven't a chance, which undermines support for democratic decision-making (see #1)
Polls make it harder to deny election results. Without a (still, somewhat) scientific polling industry getting close to the outcome in advance, it’s much easier for unscrupulous politicians to go “but my crowd size! look at this vast movement! the other side cheated!” when the election doesn’t turn out the way they want.
Basically I’d contend polling is good, or at least net good. It should be presented dryly and factually like the weather report, with as little narrativizing as possible. All the ad hoc narrative forming and storytelling about the near term political future the press likes to do, that’s the bad stuff. There should be less of that.
In a rational world, that would be true. But even well-meaning people cherry pick the polls they pay attention to, and manipulators make this many times worse. Remember, a significant number of polls are done specifically for the purpose of manipulation.
You say it "should be presented dryly." I agree, but that is not the case, and I cannot imagine a way to make it so.
Isn't the other big thing that when you consider the *source* of alpha over and above e.g. publicly available polling, it's overwhelmingly likely to emerge from insider information that thereby affects the dynamics/model of the system in illegible ways?
well you could imagine adding value by better aggregating other sources of info. social media and product-sales and foot-traffic data and whatnot. that’s not done effectively and systematically now, but maybe it could be.
On the other hand it’s not really done now because it’s devilishly hard. Paying some insiders for tips is easier if it’s legal (or if it’s illegal and you’re a crook)
> After all, a big problem with journalism is that while conveying accurate information is a big part of what journalism is for, the actual incentives for correctness and truth-telling are weak.
I’m not exactly sure how much of the point of journalism is to give people a good measure of the objective probability of something, rather than to point out things people might have been leaving out of their space of possibilities that should be included. Readers do use journalism both for setting their degrees of belief and for determining what possibilities to even have degrees of belief over. But I’m not sure it’s a failure if news coverage gives people an impression that “man bites dog” is more common than it actually is, if the alternative was people thinking “man bites dog” is impossible.
I’m skeptical that there is a true thing like the “objective probability” (particularly of single case events), and in any case, different people will need information for different reasons and will have different relative costs to being overconfident or under confident, except for players in large financial markets where stakes really are nearly linear.
My problem with news coverage is that it’s too narrative-driven. Narratives have beginnings, middles and ends and fixed human-scale structure. Politics is a huge, chaotic, emergent system with dynamics that don’t fit a tidy storyline you come up with in (say) December of the preceding election year. And yet news outlets do come up with those tidy storylines early and then cling to them for dear life (*glares at 2016 NYT*).
People are naturally prone to thinking in terms of narratives, and the sorts of people who do journalism are writerly humanistic types who (in my experience) don’t really understand there are other ways to see the world. It’s a bit of a pickle.
I’d also say this is why polling is valuable. Energetic external forcing can push the narrative groupthink off of wherever it would otherwise settle.
Yes, I think that’s right. Human minds like narrative (and this is why most fiction comes in the form of a narrative) even though the real world doesn’t have that structure.
I'm glad you've moved away from selling ad space to polymarket. I agree that, as far as intent and bias go, media outlets' selling ads to a business does not correspond to more than a business transaction and does not imply that the ad seller has altered their coverage to favor ad buyers. But market transactions, even if conducted at arms' length, can carry moral implications! Providing ad space to companies that sell harmful products facilitates the greater distribution of the products and hence their harms. It's at minimum morally praiseworthy to avoid doing that. So thanks.
Oh here's something that does add to polling IMHO. There's always a state or local election going on somewhere in the US: from filling state legislative vacancies, to state Supreme Court, to ballot initiatives, voters keep getting called on to vote at odd times between general elections. Look at the results of elections held in the US from January 2025 through August 2026 - you'll see Republicans losing in districts and states Trump won, over and over.
This is definitely separate information from polling, but it’s definitely hard to know which is more relevant for an upcoming general election. Actual votes in one election tell you something different from asking someone, “will you vote in the upcoming election, and if so, for whom?” But neither is actually the thing you care about, which is actually voting in the future election.
I'm not sure which is more relevant either. Voting in Jan 2025 - Aug 2026 elections, especially the later ones (closer to Nov 2026) probably has some relevance especially because of high "polarization". (Someone who voted D or R in Aug 2026 will probably do the same in Nov 2026.) I think both polls and recent election results are worth more attention than prediction markets since prediction markets basically just follow the polls.
Pedantic mode: you messed this one up: "stand a good chance of flipping the Senate despite unfavorable geography (unless John Fetterman flips it back)." The correct version is "stand a good chance of flipping the Senate despite unfavorable geography (unless Democrats flip EXACTLY FOUR seats and John Fetterman then flips ONE back)." Based on current polling, Democrats (or non-Republicans, to include Nebraska) have a realistic chance of flipping up to 8 Senate seats. The most Republican of these are Kansas and Nebraska. I'd say the chances of Dems winning all 8 are very low. But to get to 52 and flip the bird to Fetterman, they need to win 5 or more of the 8, which is far more likely than winning all 8. The most likely 5 are Maine, North Carolina, Ohio, than any two of Alaska, Iowa, Texas.
Seems like these thinly traded longshots could be an opportunity for something like a poor college student and an agent to make beer money.
More seriously, seems like prediction markets for the moment are just a huge win for polling being the best available source of information. That’s not to say polling doesn’t have plenty of problems, just that if better available inputs existed, prediction markets would be processing them.
Following the polls and the prediction markets is a lot of fun for journalists and politics junkies like us but at the end of the day, who cares? Campaigns (and maybe high-dollar donors) need polling information to determine investments, tactics and strategy but as for the rest of the nation, what difference would it make if all the publicly-available polls simply disappeared?
So why should we care if the prediction markets are accurate or well-calibrated? We'll know around Election Day in any case. And if we can get rid of the stories along the lines of "overwhelming victory for the Democrats but below polling expectations, causing finger-pointing and despair among party members" so much the better. Screw the expectations game; just win, baby.
If you have money at stake, sure. But what decisions would a company make a month or two before an election that would shape those long-term investments more than waiting until November? And if they make these long-term investments, say, six months before the election, I'd sell their stock because they have no idea what they're doing.
Companies need to decide whether to buy more capital goods often many months before the new production can get started. So you better believe that companies are starting some of these investments six months before an election, even if they think there’s some chance that the election could change the regulatory environment in a way that would be favorable or unfavorable for their sales.
They have to make decisions all the time. But if they're relying on information about electoral outcomes six months out from the election in all cases where there is any doubt about that outcome then they're being snookered.
I think a good model for this is that there is a “cost of intelligence”.
If you consider some financial question like, is Carnival Cruise Line overvalued or undervalued. If you get that right, it can be worth millions of dollars. So it’s worth having some very smart people do research, dig around, find non public data, and develop new insights. That generally happens, and the market prices reflect more than a brief amateur look and the fundamentals.
These political markets do not fund that level of research, so they don’t reveal that much hidden insight.
I feel like this analysis needs to calibrate “polls only” or “expert ratings” to come to the actual conclusion MY came to. How are we to know there’s not actually some marginal edge? Sort of handwavey on that side.
Calibration is not the only thing we need from prediction markets. Guessing either candidate with equal probability is perfectly calibrated.
What we also care about is Discrimination. Guessing 97% (and being well calibrated) is a lot more useful than guessing 50% (and being well calibrated.)
I feel about prediction markets the same way Joseph feels about sports. Please post about something mildly interesting or useful next time!
This is a great comment on how the prediction markets can validate poll results and even go beyond that but the markets are still very new in terms of public acceptance and awareness! The only way to make good money is to go against the grain and make a contrarian bet! But that often doesn’t work out because the bets are heavily based on evidence (ie polls) And I’m glad that you mentioned Tetlocks books, they are worth reading if you’re interested in prediction markets, especially Super Forecasters!
Personally I try to bet in markets where I think the polls might be wrong or the market is overrating something. I feel like for Dem Primaries the Party Decides is a framework that still holds up reasonably well so I made 700ish on my small bet against and lost a bit on the margin for the Michigan Primary but came pretty close there.
Prediction markets offer benefits only, and to the extent that, they can incentivize the disclosure of material non-public information to the market. The platform's fear of being soft on insider trading and the government's attempts to use standards from stock exchanges have turned them into nothing but fancy betting applications.
I mean, should you spend a bunch of money doing research to figure out who will really win so you can make money on Kalshi? Well, problem 1 is that their TOS technically ban anyone who has a formal influence over the outcome from participating and you are a voter. Problem 2 is the fuzziness of the lines on material non-public information. And hanging over all of this is the tension behind the SCOTUS ruling limiting insider trading to breach of fiduciary duty and theories of fraud against counterparties.
Yes, I know the lawyers will be able to draw some distinctions but the point is that for the behaviors that would actually make them useful -- eg incentivizing the production and sale of material information about the outcome -- have too much
legal risk to truly create the market or incentives.
Maybe we don't actually care about which google search term is the most common but imagine we did. The whole idea is that the prediction market should incentivize google to sell that data so we learn about it sooner. But even if that Google employee who bet on the most popular search terms had been contractually authorized to sample the raw search term data a month before it gets finalized and bet on it they'd still likely get prosecuted.
But how different is that from paying to run a voting survey? Where is the line? The fuzziness kills the very point.
The gamblers dissent!
1) I highly recommend Jesse Richardson's substack blog (Golden Path), as he's a 7-figure winning election bettor and has interesting things to say as a man who is fading the polling average.
2) In general, I think the real value of prediction markets in the U.S. tends to be when we have polls and early vote.
3) I mostly bet on sports, but I wrote this in direct contradiction to your point, as I think betting the polling average has actually never been less +ev than the present! https://substack.com/home/post/p-189914255.
It would seem to me that the focus of the news business ought to be on things that have actually happened, as opposed to predictions of what is yet to happen. (Or, for that matter, evaluations of those predictions.)
It is a very bad sign that so much anger is spilled over polls (or prediction markets) getting an election "wrong." With all the problems our country faces, all the signs of our decline, in effect journalists are re-focusing people on what is, at its core, entertainment. With all sorts of negative side effects:
1) Giving aid and comfort to those who would deny election results,
2) Adding to the general disillusionment with scientific inquiry as a means to know things
3) Legitimizing the slide from campaigns aiming to persuade the electorate to hyper-focusing on crucial groups or locations -- which, when you think about it, makes a mockery of our form of government
4) Inadvertently telling various viewpoints that they haven't a chance, which undermines support for democratic decision-making (see #1)
Polls make it harder to deny election results. Without a (still, somewhat) scientific polling industry getting close to the outcome in advance, it’s much easier for unscrupulous politicians to go “but my crowd size! look at this vast movement! the other side cheated!” when the election doesn’t turn out the way they want.
Basically I’d contend polling is good, or at least net good. It should be presented dryly and factually like the weather report, with as little narrativizing as possible. All the ad hoc narrative forming and storytelling about the near term political future the press likes to do, that’s the bad stuff. There should be less of that.
In a rational world, that would be true. But even well-meaning people cherry pick the polls they pay attention to, and manipulators make this many times worse. Remember, a significant number of polls are done specifically for the purpose of manipulation.
You say it "should be presented dryly." I agree, but that is not the case, and I cannot imagine a way to make it so.
Isn't the other big thing that when you consider the *source* of alpha over and above e.g. publicly available polling, it's overwhelmingly likely to emerge from insider information that thereby affects the dynamics/model of the system in illegible ways?
well you could imagine adding value by better aggregating other sources of info. social media and product-sales and foot-traffic data and whatnot. that’s not done effectively and systematically now, but maybe it could be.
On the other hand it’s not really done now because it’s devilishly hard. Paying some insiders for tips is easier if it’s legal (or if it’s illegal and you’re a crook)
> After all, a big problem with journalism is that while conveying accurate information is a big part of what journalism is for, the actual incentives for correctness and truth-telling are weak.
I’m not exactly sure how much of the point of journalism is to give people a good measure of the objective probability of something, rather than to point out things people might have been leaving out of their space of possibilities that should be included. Readers do use journalism both for setting their degrees of belief and for determining what possibilities to even have degrees of belief over. But I’m not sure it’s a failure if news coverage gives people an impression that “man bites dog” is more common than it actually is, if the alternative was people thinking “man bites dog” is impossible.
I’m skeptical that there is a true thing like the “objective probability” (particularly of single case events), and in any case, different people will need information for different reasons and will have different relative costs to being overconfident or under confident, except for players in large financial markets where stakes really are nearly linear.
My problem with news coverage is that it’s too narrative-driven. Narratives have beginnings, middles and ends and fixed human-scale structure. Politics is a huge, chaotic, emergent system with dynamics that don’t fit a tidy storyline you come up with in (say) December of the preceding election year. And yet news outlets do come up with those tidy storylines early and then cling to them for dear life (*glares at 2016 NYT*).
People are naturally prone to thinking in terms of narratives, and the sorts of people who do journalism are writerly humanistic types who (in my experience) don’t really understand there are other ways to see the world. It’s a bit of a pickle.
I’d also say this is why polling is valuable. Energetic external forcing can push the narrative groupthink off of wherever it would otherwise settle.
Yes, I think that’s right. Human minds like narrative (and this is why most fiction comes in the form of a narrative) even though the real world doesn’t have that structure.
I'm glad you've moved away from selling ad space to polymarket. I agree that, as far as intent and bias go, media outlets' selling ads to a business does not correspond to more than a business transaction and does not imply that the ad seller has altered their coverage to favor ad buyers. But market transactions, even if conducted at arms' length, can carry moral implications! Providing ad space to companies that sell harmful products facilitates the greater distribution of the products and hence their harms. It's at minimum morally praiseworthy to avoid doing that. So thanks.
Oh here's something that does add to polling IMHO. There's always a state or local election going on somewhere in the US: from filling state legislative vacancies, to state Supreme Court, to ballot initiatives, voters keep getting called on to vote at odd times between general elections. Look at the results of elections held in the US from January 2025 through August 2026 - you'll see Republicans losing in districts and states Trump won, over and over.
This is definitely separate information from polling, but it’s definitely hard to know which is more relevant for an upcoming general election. Actual votes in one election tell you something different from asking someone, “will you vote in the upcoming election, and if so, for whom?” But neither is actually the thing you care about, which is actually voting in the future election.
I'm not sure which is more relevant either. Voting in Jan 2025 - Aug 2026 elections, especially the later ones (closer to Nov 2026) probably has some relevance especially because of high "polarization". (Someone who voted D or R in Aug 2026 will probably do the same in Nov 2026.) I think both polls and recent election results are worth more attention than prediction markets since prediction markets basically just follow the polls.
Pedantic mode: you messed this one up: "stand a good chance of flipping the Senate despite unfavorable geography (unless John Fetterman flips it back)." The correct version is "stand a good chance of flipping the Senate despite unfavorable geography (unless Democrats flip EXACTLY FOUR seats and John Fetterman then flips ONE back)." Based on current polling, Democrats (or non-Republicans, to include Nebraska) have a realistic chance of flipping up to 8 Senate seats. The most Republican of these are Kansas and Nebraska. I'd say the chances of Dems winning all 8 are very low. But to get to 52 and flip the bird to Fetterman, they need to win 5 or more of the 8, which is far more likely than winning all 8. The most likely 5 are Maine, North Carolina, Ohio, than any two of Alaska, Iowa, Texas.
Seems like these thinly traded longshots could be an opportunity for something like a poor college student and an agent to make beer money.
More seriously, seems like prediction markets for the moment are just a huge win for polling being the best available source of information. That’s not to say polling doesn’t have plenty of problems, just that if better available inputs existed, prediction markets would be processing them.
Following the polls and the prediction markets is a lot of fun for journalists and politics junkies like us but at the end of the day, who cares? Campaigns (and maybe high-dollar donors) need polling information to determine investments, tactics and strategy but as for the rest of the nation, what difference would it make if all the publicly-available polls simply disappeared?
So why should we care if the prediction markets are accurate or well-calibrated? We'll know around Election Day in any case. And if we can get rid of the stories along the lines of "overwhelming victory for the Democrats but below polling expectations, causing finger-pointing and despair among party members" so much the better. Screw the expectations game; just win, baby.
It also matters for companies who are planning eg, long-term investments and want to know what the likely regulatory regime will be months in advance.
If you have money at stake, sure. But what decisions would a company make a month or two before an election that would shape those long-term investments more than waiting until November? And if they make these long-term investments, say, six months before the election, I'd sell their stock because they have no idea what they're doing.
Companies need to decide whether to buy more capital goods often many months before the new production can get started. So you better believe that companies are starting some of these investments six months before an election, even if they think there’s some chance that the election could change the regulatory environment in a way that would be favorable or unfavorable for their sales.
They have to make decisions all the time. But if they're relying on information about electoral outcomes six months out from the election in all cases where there is any doubt about that outcome then they're being snookered.
Data centers?
It doesn't take too much sophisticated analysis or data gathering to note that people are 80/20 against data centers.
I think a good model for this is that there is a “cost of intelligence”.
If you consider some financial question like, is Carnival Cruise Line overvalued or undervalued. If you get that right, it can be worth millions of dollars. So it’s worth having some very smart people do research, dig around, find non public data, and develop new insights. That generally happens, and the market prices reflect more than a brief amateur look and the fundamentals.
These political markets do not fund that level of research, so they don’t reveal that much hidden insight.
I feel like this analysis needs to calibrate “polls only” or “expert ratings” to come to the actual conclusion MY came to. How are we to know there’s not actually some marginal edge? Sort of handwavey on that side.