What we are witnessing is the culmination of linguistic and empirical nihilism. Truth does not matter. The mere act of knowing does not matter. Whatever is rhetorically convenient for people assert to in the moment is true until it is no longer useful.
I had this interaction several times during the Biden administration:
Person: "Biden has cut oil production in the US."
Me: "US oil production is close to all-time high and it's been going up under Biden, here's a chart from the U.S. Energy Information Administration."
Person: "They're faking the numbers."
I've had this happen in-person and with people who are not crazy or unintelligent; one had a PhD in engineering. It's really strange, we're at the point in partisanship that it's almost impossible to have a conversation because trust and concern for truth are close to zero.
Part of the problem is that the Biden administration itself and its surrogates didn't want to talk about booming oil production because they didn't want to anger climate activists. Biden's opponents also didn't want to bring it up because they correctly reasoned that doing so would help Biden. And of course, most people aren't spending their days combing through OPEC data on crude oil markets and crude oil production isn't really a front-page topic for most news media.
So in the end, the only people really aware of such data are pro-Biden liberals waiting to make snarky replies to conservatives in social media discourse.
The immediate jump to "the government is fabricating data" is what makes these interactions noteworthy to me. That shows a huge lack of trust. Partisanship is breaking brains much more so than it used to.
It's all vibes. Biden puts out anti-fossil-fuel vibes so everything is seen through that lens. So much in our lives is now divorced from (or at least insulated from) policy that we usually can't see the effects directly.
Trump is now reminding us that choices really do have consequences.
I wish Biden had openly campaigned on abundant energy and energy independence rather than the climate crisis shit. That would have made solar much more appealing.
I don't think it's just vibes; or it's stronger than just vibes because the vibes are backed by policy. He immediately revoked the Keystone XL pipeline permits and then later paused LNG export approvals -- caving to the irrational demands of "the groups".
The saying, "'Engineer' in bio," is a thing for a reason! (Actually, it isn't, but the last month of economic discourse on Twitter has made me think it should be.)
I saw a meme that called it Critical Economic Theory and I LOLed. I'm not proud of it but I'm filled with schadenfreude watching these idiots torture themselves trying to explain the brilliant 4D chess we're all not witnessing.
This is an excellent overview and could influence the template of what could be called Project 2029. A key aspect of the (inshallah) 2029 not Trump administration will be thoroughly documenting the damage done now, and taking time over the next four years to plan out what must be reconstituted and what must be reformed along with reconstitution.
I superlike™️ this concept (Project 2029/Reconstruction). We should not let The Groups fill the void (not that most of them are focused on winning, let alone actual policy).
Democrats are going to need to divorce themselves with their obsession over process, reviews, committees, and other such nonsense and just focus on how to most quickly get the outcomes we want.
People are decrying the evisceration of a whole apparatus that took decades to build, so that is why we need to rebuild it with haste and agency like FDR did. Need housing, it should take 3 months between application and breaking ground to do infill housing. It is insane how we let things get mired in red tape when the property is freaking vacant. New Englanders don't want transmission lines. Ask them what they would need for them to accept the project, if they don't give you any "yes" condition use eminent domain and make sure the law strips them of asinine litigation rights. Strip out all fucking nonsense about equity and justice when it comes to building fucking things like solar panels with government money.
All that matters is does the project pencil out and will it get built.
Umm, did you read this post? It’s all about setting up process, reviews, and committees. We want systematic processes for gathering data, not just rushing to grab whatever data you can get this month opportunistically. We want committees to review this data in systematic ways, not whatever seems most efficient this month.
My point is if everything is broken we should not be slow in rebuilding it anew. I don't want a repeat of the IRA or BIL where all this money gets allocated and then doesn't get used because of asinine red tape. We are in our current situation because Biden and Democrats didn't take Trump's threat seriously. They let him walk for years and build his criminal conspiracy against this country, all the while they dragged their asses over purely discretionary processes.
I absolutely agree about that. But it is also extremely important to remember that there are many processes, reviews, and committees that are important for a reason. Don’t let yourself get negatively polarized against these things just because a lot of them are done badly. We shouldn’t be for *or* against them in the abstract, any more than we are for or against “regulation”.
"Democrats are going to need to divorce themselves with their obsession over process, reviews, committees, and other such nonsense and just focus on how to most quickly get the outcomes we want."
If we go overboard with that, we just turn into Trump. There has to be a balance.
I was always partial to the term "risorgimento" attached to the unification of Italy. Being a parochial nation, we'd of course need to go with "American Rebirth"
Not just economic data is at risk. Public health data, demographic data, crime statistics ... It's no coincidence that one of the gun industry's most effective strategies was to handicap the collection and dissemination of statistics related to gun-related crime. It's also no coincidence that are gold-standard data resources are being systematically dismantled.
Nice one! Matt refers to decision making based on accurate data but there are also contractual clauses that refer to official statistics (for ex.: I can't and wouldn't particularly want to change the rent I charge my tenants but, per our contract, it is indexed to the price level and my tenants and I count on an accurate and fair measurement).
I don’t think people understand how unusual it is for republican administration to damage data collection like this. Especially economic statistics. The demolition of FHFA’s research group is particularly wild given who stood it up (a Cato guy) and why (long term GOP concerns about bad data from housing advocates).
There’s a longer post about how there’s a bipartisan problem of people (almost always project managers and MPPs, sometimes lawyers) who think they know the answer and feel like researchers just exist to undermine them.
My physician wife had to get some data from the CDC the other day. The first thing she did was check the date on the info, it was 2017, so she had confidence in it.
When do we start looking at data having vintages, like wine. Oh that's a 2025, ooh not a good year: bad taste, suspicious provenance. Better select the 2008, even if it's contradicted by the '25.
“ Each new series of estimates (referred to as a "vintage") is revised annually beginning with the date of the most recent decennial census to incorporate the latest administrative record data, geographic boundaries, and methodology. The vintage year (e.g., V2024) refers to the final year of the time series.”
"An Administration that wants a less wasteful, more efficient government needs accurate and comprehensive federal statistics."
The right is far too brain-drained to want this in any but the most amorphous, vague terms.
They pay it lip service but even before the tariff debacles most of their major corporate allies were essentially throwing up their hands and saying, "Why are you firing everyone we need to rewrite the rules to be simpler?"
We really are genuinely operating in "Idiocracy" territory, but Donald Trump is actually worse than President Camacho (who both seemed to be relatively non-corrupt and, more importantly for this purpose, was willing to listen to someone with more expertise than himself to try to learn from them).
I just read the scream of horror at Trump's depradations written by Bret Stephens (https://www.nytimes.com/2025/04/08/opinion/trump-deporation-america.html.) Yes, he's an easy target, but still. The New York Times helpfully gave links to other recent Stephens' column, including the one he wrote right after Trump won, devoted to his smacking his lips over the Democratic comeuppance. Yes yes, he very very reluctantly said he would vote for Harris, with gritted teeth, but you could feel his pleasure in writing things like:
"Why did Harris lose? . . . her foolish designation of Trump as a fascist, which, by implication, suggested his supporters were themselves quasi-fascist. . .
"liberals thought that the best way to stop Trump was to treat him not as a normal, if obnoxious, political figure with bad policy ideas but as a mortal threat to democracy itself. . "
Silly Democrats.
And just to tie it to Jed's important and welcome post today, I loved this choice bit from the same Stephens' column about the Democrats' feckless attempts to paint a rosier picture of the economy last year: "I’ve lost track of the number of times liberal pundits have attempted to steer readers to arcane data from the St. Louis Federal Reserve to explain why Americans should stop freaking out over sharply higher prices of consumer goods or the rising financing costs on their homes and cars."
This is an important issue, and a scary one to contemplate, so I'm glad to have read this. Another confounding factor is the other institution in this country that could fill the void is higher education, but we see how that's going. Cornell getting hit with a ridiculous $1 Billion "fine" for nonsense violations is bonkers... I can tell you they do a lot of great work on population statistics, utilizing Census data (among other data sets), that's almost certainly going to be cut. What if MIT puts out an inflation estimate that doesn't align with the Trump numbers?
I was going to ask if anybody could tell me what the fuck is going on with tariffs, because I just read the NYT and I can't quite make it out.
Instead, I'll try to ask a more narrow question. Can anyone tell me, for any or each of the following countries, Canada, Mexico, Japan, whether we have or have not imposed tariffs on them?
Important that you wrote about in such depth and with clarity. Statistical issues are like termites in the woodwork (my favorite analogy from my long time mentor at Brookings, Charlie Schultze). And what you describe is a whole lot of termites
I would wonder more about this data to begin with. Financial markers pay 3rd party firms to arrive at more accurate data so they can arb the errors in federal data.
The ag planting data for crop insurance is at least 10% off of actual. All efforts to improve it have been subverted for years, because trial bar wanted to use it for various litigation efforts.
Inflation data fails to measure inflation correctly or the right market basket. The last administration kept saying inflation was under control. Was it? There is no proof indexes for COLA are accurate. We are likely not close on properly measuring productivity.
Amongst governments our data may be better. But that achievement may be far off accurate.
The most accurate data is likely private feeds to hedge fund markets. Getting that right or wrong matters.
You say inflation isn’t being measured “correctly” or the “right market basket”. What does that mean? It seems to me there are many things one can measure here, some of which may be what you care about for one purpose and some for another. I don’t think one of them is *the* correct measure. But what you want is a trustworthy measure of the same thing for years on end in a public and verifiable way, so that even if you care about a different basket, you can have some sense of how what you care about differs from it.
Using private data is not trustworthy for anyone other than the people who make that data - why would a competitor trust the public releases, and how would we know if the underlying measure they are getting is changing?
When you have a productivity boost like uber, eliminating the need for a car, you have a TFP productivity boost that is deflationary. But if government measures a car as core inflation, it misguided.
Someone selling private gap data to a hedge fund has no reason to misguide. If they did the hedge fund would lose money and never buy again.
Corporations buy and assemble private data all the time to get more accurate market insight. There are a fre very interesting market basket data sets that are much more accurate than the federal data. ADP data is another example. Another way to ask the question is who relies on federal data? In ag, ag credit system does because they have to by law. But commodity traders independently assemble the data because they know the USDA data is wrong. Everyone in industry knows that. Not a secret. If you traded based on federal data you would be wrong 100% of the time. Frankly it's part of reason Cargill and ADM have so much power. And they like it that way. As do like JPMs treasury desk.
I still think it's misleading to say the federal data is "wrong". That's like saying that it's "wrong" for the US census to count the number of immigrants that live in various places. For some purposes, you absolutely want a different number than the one the government is tracking. The people who are willing to pay a lot to track precise numbers, will probably want to track particular ones that differ from what the government is tracking.
But the government numbers are for the 98% of people who *don't* have enough reason to track these things that they want to spend large amounts of money on it. And for those people, what is most relevant is that there is *something* that is being tracked systematically and transparently, in a way that allows you to compare it month to month and year to year, ideally with easy ways to supplement it if something slightly different is better for your purposes.
I don't complain too much that the national weather service is tracking the temperature at city hall, four miles from my house - it's still pretty good, even though it's quite often several degrees off from the temperature at my house.
If it doesn't matter if the data is accurate than what is the point of Matt's post. I doubt people have a specific utility for an semi accurate GDP number except maybe for debates on X or Bluesky.
It matters if the data are an accurate measure of the thing they have been measuring for years. It doesn't matter so much if it's precisely the thing that some particular hedge fund or commodity trader is interested in.
We probably can measure the economy more consistently, weather perhaps less though.
Most weather reports depend on the noaa models. A few years ago however we needed a new super computer and US was afraid to buy the one from China. So we waited for IBM to build one. For those 2 or 3 years weather reports in US moved to the European model which was better than NOAA. We knew the US model was weak. There is no reason to think a commercial or government model is inherently better because of the economic incentives. Though economic incentives do motivate improvements.
There is substantial rent seeking in government data. Food pyramid is one good example of government data gone wrong.
Every authoritarian regime makes up economic data to suit the needs of the ruler. The Soviet Union did it. China does it. North Korea does it. In the book 1984, Big Brother did it. And, a dictatorship of the United States would undoubtfully do it too. It should not be surprising.
What we are witnessing is the culmination of linguistic and empirical nihilism. Truth does not matter. The mere act of knowing does not matter. Whatever is rhetorically convenient for people assert to in the moment is true until it is no longer useful.
Ughh.
I had this interaction several times during the Biden administration:
Person: "Biden has cut oil production in the US."
Me: "US oil production is close to all-time high and it's been going up under Biden, here's a chart from the U.S. Energy Information Administration."
Person: "They're faking the numbers."
I've had this happen in-person and with people who are not crazy or unintelligent; one had a PhD in engineering. It's really strange, we're at the point in partisanship that it's almost impossible to have a conversation because trust and concern for truth are close to zero.
Part of the problem is that the Biden administration itself and its surrogates didn't want to talk about booming oil production because they didn't want to anger climate activists. Biden's opponents also didn't want to bring it up because they correctly reasoned that doing so would help Biden. And of course, most people aren't spending their days combing through OPEC data on crude oil markets and crude oil production isn't really a front-page topic for most news media.
So in the end, the only people really aware of such data are pro-Biden liberals waiting to make snarky replies to conservatives in social media discourse.
The immediate jump to "the government is fabricating data" is what makes these interactions noteworthy to me. That shows a huge lack of trust. Partisanship is breaking brains much more so than it used to.
It's all vibes. Biden puts out anti-fossil-fuel vibes so everything is seen through that lens. So much in our lives is now divorced from (or at least insulated from) policy that we usually can't see the effects directly.
Trump is now reminding us that choices really do have consequences.
I wish Biden had openly campaigned on abundant energy and energy independence rather than the climate crisis shit. That would have made solar much more appealing.
We might have also gotten the public option he promised, which I still feel was the biggest failure of his administration.
https://jacobin.com/2022/08/joe-biden-public-option-health-care-insurance-subsidies
I don't think it's just vibes; or it's stronger than just vibes because the vibes are backed by policy. He immediately revoked the Keystone XL pipeline permits and then later paused LNG export approvals -- caving to the irrational demands of "the groups".
https://www.npr.org/sections/inauguration-day-live-updates/2021/01/20/958823085/biden-order-blocks-keystone-xl-pipeline
https://www.reuters.com/business/energy/biden-pauses-approval-new-lng-export-projects-win-climate-activists-2024-01-26/
"one had a PhD in engineering"
The saying, "'Engineer' in bio," is a thing for a reason! (Actually, it isn't, but the last month of economic discourse on Twitter has made me think it should be.)
This is one of the things keeping me up at night
This is all part of a greater moral rot in society.
No, this one is pretty much all on Trump, MAGA and the supine Republicans.
I saw a meme that called it Critical Economic Theory and I LOLed. I'm not proud of it but I'm filled with schadenfreude watching these idiots torture themselves trying to explain the brilliant 4D chess we're all not witnessing.
This is an excellent overview and could influence the template of what could be called Project 2029. A key aspect of the (inshallah) 2029 not Trump administration will be thoroughly documenting the damage done now, and taking time over the next four years to plan out what must be reconstituted and what must be reformed along with reconstitution.
Maybe we call the plan Reconstruction.
I superlike™️ this concept (Project 2029/Reconstruction). We should not let The Groups fill the void (not that most of them are focused on winning, let alone actual policy).
Democrats are going to need to divorce themselves with their obsession over process, reviews, committees, and other such nonsense and just focus on how to most quickly get the outcomes we want.
People are decrying the evisceration of a whole apparatus that took decades to build, so that is why we need to rebuild it with haste and agency like FDR did. Need housing, it should take 3 months between application and breaking ground to do infill housing. It is insane how we let things get mired in red tape when the property is freaking vacant. New Englanders don't want transmission lines. Ask them what they would need for them to accept the project, if they don't give you any "yes" condition use eminent domain and make sure the law strips them of asinine litigation rights. Strip out all fucking nonsense about equity and justice when it comes to building fucking things like solar panels with government money.
All that matters is does the project pencil out and will it get built.
Umm, did you read this post? It’s all about setting up process, reviews, and committees. We want systematic processes for gathering data, not just rushing to grab whatever data you can get this month opportunistically. We want committees to review this data in systematic ways, not whatever seems most efficient this month.
My point is if everything is broken we should not be slow in rebuilding it anew. I don't want a repeat of the IRA or BIL where all this money gets allocated and then doesn't get used because of asinine red tape. We are in our current situation because Biden and Democrats didn't take Trump's threat seriously. They let him walk for years and build his criminal conspiracy against this country, all the while they dragged their asses over purely discretionary processes.
I absolutely agree about that. But it is also extremely important to remember that there are many processes, reviews, and committees that are important for a reason. Don’t let yourself get negatively polarized against these things just because a lot of them are done badly. We shouldn’t be for *or* against them in the abstract, any more than we are for or against “regulation”.
I am just an old and grumpy Fed worried about getting RIFed.
"Democrats are going to need to divorce themselves with their obsession over process, reviews, committees, and other such nonsense and just focus on how to most quickly get the outcomes we want."
If we go overboard with that, we just turn into Trump. There has to be a balance.
I was always partial to the term "risorgimento" attached to the unification of Italy. Being a parochial nation, we'd of course need to go with "American Rebirth"
Not just economic data is at risk. Public health data, demographic data, crime statistics ... It's no coincidence that one of the gun industry's most effective strategies was to handicap the collection and dissemination of statistics related to gun-related crime. It's also no coincidence that are gold-standard data resources are being systematically dismantled.
Nice one! Matt refers to decision making based on accurate data but there are also contractual clauses that refer to official statistics (for ex.: I can't and wouldn't particularly want to change the rent I charge my tenants but, per our contract, it is indexed to the price level and my tenants and I count on an accurate and fair measurement).
Right on. We’ve used the CPI to calculate part of the raise we give people who work for us
I don’t think people understand how unusual it is for republican administration to damage data collection like this. Especially economic statistics. The demolition of FHFA’s research group is particularly wild given who stood it up (a Cato guy) and why (long term GOP concerns about bad data from housing advocates).
There’s a longer post about how there’s a bipartisan problem of people (almost always project managers and MPPs, sometimes lawyers) who think they know the answer and feel like researchers just exist to undermine them.
My physician wife had to get some data from the CDC the other day. The first thing she did was check the date on the info, it was 2017, so she had confidence in it.
When do we start looking at data having vintages, like wine. Oh that's a 2025, ooh not a good year: bad taste, suspicious provenance. Better select the 2008, even if it's contradicted by the '25.
It’s actually standard terminology already!
“ Each new series of estimates (referred to as a "vintage") is revised annually beginning with the date of the most recent decennial census to incorporate the latest administrative record data, geographic boundaries, and methodology. The vintage year (e.g., V2024) refers to the final year of the time series.”
https://www.census.gov/programs-surveys/popest/data/tables.html
SB commenters teach me something new every day.
"An Administration that wants a less wasteful, more efficient government needs accurate and comprehensive federal statistics."
The right is far too brain-drained to want this in any but the most amorphous, vague terms.
They pay it lip service but even before the tariff debacles most of their major corporate allies were essentially throwing up their hands and saying, "Why are you firing everyone we need to rewrite the rules to be simpler?"
Deregulation requires state capacity.
Great post! Reliably collected data gives us a window to the truth. Clearly, that is a threat to this administration.
It is terribly disheartening that near universal condemnation by experts is now worn like a badge of honor by the White House. Knowledge is the enemy.
We really are genuinely operating in "Idiocracy" territory, but Donald Trump is actually worse than President Camacho (who both seemed to be relatively non-corrupt and, more importantly for this purpose, was willing to listen to someone with more expertise than himself to try to learn from them).
I just read the scream of horror at Trump's depradations written by Bret Stephens (https://www.nytimes.com/2025/04/08/opinion/trump-deporation-america.html.) Yes, he's an easy target, but still. The New York Times helpfully gave links to other recent Stephens' column, including the one he wrote right after Trump won, devoted to his smacking his lips over the Democratic comeuppance. Yes yes, he very very reluctantly said he would vote for Harris, with gritted teeth, but you could feel his pleasure in writing things like:
"Why did Harris lose? . . . her foolish designation of Trump as a fascist, which, by implication, suggested his supporters were themselves quasi-fascist. . .
"liberals thought that the best way to stop Trump was to treat him not as a normal, if obnoxious, political figure with bad policy ideas but as a mortal threat to democracy itself. . "
Silly Democrats.
And just to tie it to Jed's important and welcome post today, I loved this choice bit from the same Stephens' column about the Democrats' feckless attempts to paint a rosier picture of the economy last year: "I’ve lost track of the number of times liberal pundits have attempted to steer readers to arcane data from the St. Louis Federal Reserve to explain why Americans should stop freaking out over sharply higher prices of consumer goods or the rising financing costs on their homes and cars."
https://www.nytimes.com/2024/11/06/opinion/donald-trump-defeat-democrats.html
Well, under Trump we won't have to worry too much about arcane data from FRED anymore.
This is an important issue, and a scary one to contemplate, so I'm glad to have read this. Another confounding factor is the other institution in this country that could fill the void is higher education, but we see how that's going. Cornell getting hit with a ridiculous $1 Billion "fine" for nonsense violations is bonkers... I can tell you they do a lot of great work on population statistics, utilizing Census data (among other data sets), that's almost certainly going to be cut. What if MIT puts out an inflation estimate that doesn't align with the Trump numbers?
I was going to ask if anybody could tell me what the fuck is going on with tariffs, because I just read the NYT and I can't quite make it out.
Instead, I'll try to ask a more narrow question. Can anyone tell me, for any or each of the following countries, Canada, Mexico, Japan, whether we have or have not imposed tariffs on them?
I don't think anyone, including Donald Trump himself, could give you a clear answer
Important that you wrote about in such depth and with clarity. Statistical issues are like termites in the woodwork (my favorite analogy from my long time mentor at Brookings, Charlie Schultze). And what you describe is a whole lot of termites
I would wonder more about this data to begin with. Financial markers pay 3rd party firms to arrive at more accurate data so they can arb the errors in federal data.
The ag planting data for crop insurance is at least 10% off of actual. All efforts to improve it have been subverted for years, because trial bar wanted to use it for various litigation efforts.
Inflation data fails to measure inflation correctly or the right market basket. The last administration kept saying inflation was under control. Was it? There is no proof indexes for COLA are accurate. We are likely not close on properly measuring productivity.
Amongst governments our data may be better. But that achievement may be far off accurate.
The most accurate data is likely private feeds to hedge fund markets. Getting that right or wrong matters.
You say inflation isn’t being measured “correctly” or the “right market basket”. What does that mean? It seems to me there are many things one can measure here, some of which may be what you care about for one purpose and some for another. I don’t think one of them is *the* correct measure. But what you want is a trustworthy measure of the same thing for years on end in a public and verifiable way, so that even if you care about a different basket, you can have some sense of how what you care about differs from it.
Using private data is not trustworthy for anyone other than the people who make that data - why would a competitor trust the public releases, and how would we know if the underlying measure they are getting is changing?
When you have a productivity boost like uber, eliminating the need for a car, you have a TFP productivity boost that is deflationary. But if government measures a car as core inflation, it misguided.
Someone selling private gap data to a hedge fund has no reason to misguide. If they did the hedge fund would lose money and never buy again.
Corporations buy and assemble private data all the time to get more accurate market insight. There are a fre very interesting market basket data sets that are much more accurate than the federal data. ADP data is another example. Another way to ask the question is who relies on federal data? In ag, ag credit system does because they have to by law. But commodity traders independently assemble the data because they know the USDA data is wrong. Everyone in industry knows that. Not a secret. If you traded based on federal data you would be wrong 100% of the time. Frankly it's part of reason Cargill and ADM have so much power. And they like it that way. As do like JPMs treasury desk.
I still think it's misleading to say the federal data is "wrong". That's like saying that it's "wrong" for the US census to count the number of immigrants that live in various places. For some purposes, you absolutely want a different number than the one the government is tracking. The people who are willing to pay a lot to track precise numbers, will probably want to track particular ones that differ from what the government is tracking.
But the government numbers are for the 98% of people who *don't* have enough reason to track these things that they want to spend large amounts of money on it. And for those people, what is most relevant is that there is *something* that is being tracked systematically and transparently, in a way that allows you to compare it month to month and year to year, ideally with easy ways to supplement it if something slightly different is better for your purposes.
I don't complain too much that the national weather service is tracking the temperature at city hall, four miles from my house - it's still pretty good, even though it's quite often several degrees off from the temperature at my house.
If it doesn't matter if the data is accurate than what is the point of Matt's post. I doubt people have a specific utility for an semi accurate GDP number except maybe for debates on X or Bluesky.
It matters if the data are an accurate measure of the thing they have been measuring for years. It doesn't matter so much if it's precisely the thing that some particular hedge fund or commodity trader is interested in.
We probably can measure the economy more consistently, weather perhaps less though.
Most weather reports depend on the noaa models. A few years ago however we needed a new super computer and US was afraid to buy the one from China. So we waited for IBM to build one. For those 2 or 3 years weather reports in US moved to the European model which was better than NOAA. We knew the US model was weak. There is no reason to think a commercial or government model is inherently better because of the economic incentives. Though economic incentives do motivate improvements.
There is substantial rent seeking in government data. Food pyramid is one good example of government data gone wrong.
Every authoritarian regime makes up economic data to suit the needs of the ruler. The Soviet Union did it. China does it. North Korea does it. In the book 1984, Big Brother did it. And, a dictatorship of the United States would undoubtfully do it too. It should not be surprising.
How does one measure the value or impact of these datasets?
Perhaps by size of hedge markets on indexes dependent on the data. Interesting question.