Official data is flawed, epistemic nihilism is worse

Measuring the economy in real time is just difficult

Over the weekend, a handful of people from the tech sector, namely David Sacks and Balaji Srinivasan, seemed to accuse the Bureau of Labor Statistics of cooking the books on jobs numbers. This drew substantial (and substantive) criticism from Econ Twitter before the conversation devolved into name-calling.

I think the tech sector folks are pretty clearly wrong here, primarily because if they weren’t, we would already know. “The media” functions much better than its critics claim, and practically all of the outlets that are fairly characterized as biased toward the left would nevertheless love to publish a book-cooking scoop from a whistleblower at the Department of Labor or Commerce.

Many on the right are psychologically and politically invested in anti-media narratives, but I promise that if you log off for 10 minutes, take a few deep breaths, and think this through logically, you’ll realize that in whatever sense you think The New York Times and CNN and the Washington Post and ABC News and Reuters are in the tank for Joe Biden, it’s just not true that their reporters would deliberately bury a huge career-advancing scoop. It may also occur to you, in this calm and contemplative headspace, that if for some reason those reporters did try to bury the story, the whistleblower could take it to Fox or the Wall Street Journal or Joe Rogan or any one of the many, many other high-profile, extremely successful media outlets that you arbitrarily exclude from your account of “the media” when you characterize it as in the tank for Biden. All that stuff you know about that the media doesn’t want you to know about? You found out about it from other media outlets, the very same outlets that would blow up any effort to cover up an economic data conspiracy.

But while the bailey version of this argument is dumb, the motte version is underrated. It drives me crazy that the federal government reports homicide statistics with what is optimistically an 18-month lag, leaving us with no better option than to squint at Jeff Asher’s big city data and guess what’s happening (murder seems to be down 10% from last year, for the record). With the economy, the government does a much better job of releasing timely information, which I think is good. But measuring macroeconomic aggregates is objectively more difficult than tallying dead bodies, and it’s important to be realistic about how solid this information really is.

One should not run around accusing civil servants of participating in implausible conspiracies. I also think it’s fundamentally wrong to frame this as a question of the government somehow being inept or incapable of doing things. The lack of timely and comprehensive murder data really is a huge failure of state capacity and something that policymakers should address. But America’s macroeconomic data is really good by global standards, it is routinely used by private sector actors even as they supplement it with their own, and it’s revised over time as more information becomes available. Questions like “what is the total value of goods and services produced in the United States?” are simply difficult to answer, and questions like “what share of the quarter-to-quarter change in that value are due to changes in prices rather than changes in the quantity of goods?” pose both theoretical and empirical challenges.

It is always important to keep in mind that the official data might be off in either direction, and if you believe some non-government source of information gives you insight into which direction that is, that’s a totally reasonable way to go about generating a hypothesis. That said, as an intellectual discipline, you should also keep in mind that there are a lot of people at quant funds and investment banks scouring the world for an edge, and while these traders may not be perfect, they are putting real money on the line.

It’s also important to maintain some internal consistency in your hypothesis. The economy might be running hotter than the BLS says or it might be running cooler, but it’s quite unlikely that things are worse than they appear in all directions simultaneously.

The simplest possible demonstration that official data is wrong

If you want to convince the world that American economic data suffers from accuracy problems, the best way to do it is probably to share a chart like this one.