Been doing one or more of the three since 1987. Not especially good myself, but I was trained by some of the best and it gave me opportunities I had only dreamed of and a body of work to look back on with pride.
I don't think Kolko is making this mistake, but many commenters seem to, so I will correct it. The need for the kind work described here is not in opposition to the kind of work that often gets published and is less directly useful for government. In fact, literature review and modeling are both reliant on prior literature. The problem is not that novel work that requires a PhD to appreciate isn't valuable, it's that we also need to value the work described here.
How about some research on how bad our system is for getting national data on things like crime or medical statistics. Ideally with some ways to fix it.
I appreciate the insights on what types of research are useful to government agencies for making policy, but there seems to be an implicit assumption, intended or not, of "it should be someone else's responsibility to make sure this research gets funded, completed, and published in a timely fashion." I get it's not easy to change how an institution like the Department of Commerce operates, but... no. If you need additional info to do your job well, then it is also your job to make sure that there is some sort of RFP with a timely process for getting the project scope, funded, and done to produce that info. That may very well mean funding academics, it may not, but either way it's not their responsibility to change how they operate based on your needs unless you make it worth their while. I know, I know, "Fix how government buys things it needs" is a quick way to get bogged down in an endless quagmire, but that doesn't make it less true.
Very interesting to hear the inside scoop on how the sausages are made! The Journal of Economic Literature is indeed a godsend. Are you aware of the Stanford Impact Labs? The intention is precisely to fill this need for research that is boring by journal standards, but necessary and insightful for policy-makers. I believe the director, Jeremy Weinstein, had the same experience as you of having worked in government and found the decision-making to be shockingly unempirical. https://impact.stanford.edu/
I think this issue of bridging between experts (researchers, academics, etc.) and policymakers and policymaking is a very general one. In health care there is a similar disconnect between the available policy research and the use of it by government. I only recall it vauely but I recall an analysis by a med student of all congressional testimony and debate leading up to the demonstration that came to be known as Medicare Health Support. THere was basically no involvment of any health expert, policy research, or even just clinical practitioner in the discussions. The iniative rappidly turned into a give away to "disease management companies" and after hudreds of millions of dollars had been spent disolved into regret, recrimination, and law suits. https://www.nejm.org/doi/full/10.1056/nejmsa1011785 This is what encouraged me to set up and initially fund (using OPM) a program to bring experts (mostly academics) into government for exposure to the policy process https://www.healthandagingpolicy.org/ and policy makers exposure to them. Mostly it seems to have worked well, even though I'm not sure our policy is better yet.
And perhaps worse, those within the policy making sphere don't seem able to accurately evaluate their own efforts or even remember them sometimes. If we aren't going to learn from what we've tried before why do we think we will get better outcomes? As administrations change, as embarassing news is hidden away, and empirical results are overwhelmed by idiology, we keep reinventing the flat tire.
Great guest. Interesting observation: "Cleaning up contaminated sites for redevelopment creates jobs at a much lower cost per job than job training, which in turn is much more cost-effective than giving businesses tax breaks or grants to create jobs."
I tend to hear this one about retraining diminished by cynicism from the plural of anecdotes: "There's no jobs for the retrained.","The trainees aren't interested in learning, they only want their old job back, they're just there to qualify for the check."
In any case, to the extent this data you say has checked out, I hoped it's not just accounted for and acted upon in federal programs, but widely circulated to states, counties, municipalities, and tribal entities, because the last, less efficient alternative of tax breaks and incentives shaves away fiscal health, and pits localities against each other in a race to the bottom and is an open invitation to self-dealing.
Wait till he sees how useless most academic legal publishing is to practicing lawyers, judges, legislators, or really anyone else except other legal academics.
As a current student staffer on a law journal…legal scholarship is some of the most pointless I’ve ever seen. Legal academia is desperately overdue for some disruption.
As a counterpoint to what I said above, here is an example of academic legal publishing of a type that could be useful to lawmakers (judicial and legislative), because it provides an empirical description of the current law on the ground, in a controversial and contested area where there are competing claims about what and how stable the current law actually is. And where some of those in the debate have an incentive to overstate or misconstrue current law to gain support for legislation to change the law. Note it was not published in a law school law review.
I think there is a LOT of low-hanging fruit out there if legal scholars are willing to get more empirical. But they face serious headwinds such as: it’s hard to get data on courts; law tends to draw people from non-technical backgrounds who are uncomfortable with/distrustful of empiricism; student-run journals (and the profession more broadly) value prestigious credentials over content; some legal philosophies explicitly reject empiricism; etc.
Do Supreme Court justices, who say what the Constitution is, care at all what all the brilliant constitutional experts in law schools have to say, other than when it confirms their priors?
Just a shout out to two groups the author neglected who do practical econ policy research so well on food and ag: the USDA's Economic Research Service and at the universities, the Extension service.
I finished my PhD last year and then moved into government work, and this piece captured a lot of what I think is silly about academic research. At the end of the day the vast majority of it isn’t useful for policy work, no matter how many times they say it has important policy implications in the intro. But I agree that you can’t publish in a top journal or get tenure track doing most of the work you’re describing, unless you’ve already “made it.”
I didn’t say the purpose of Econ academia is to be useful for policy work. I said much of it claims to be useful for policy work and it isn’t. And I find that aspect of it to be silly. If the field was a little self aware, then I would agree more with what you’re saying.
Google shows that the phrase “Biden Administration” is 67 times more frequent than “Biden-Harris Administration.” So it’s a little weird, but not unprecedented.
When you work in the administration, it’s respectful and considered good practice to refer to both the President and the VP as the name of the administration. Think you might be reading into this one a bit much.
You must not have been paying attention to the 2020 primary. Biden called it the “Obama-Biden administration all the time; “Biden-Harris” is a continuation of that.
On a related matter, there are economic data that the Government should produce:
1. Treasury should issue TIPS at less than 5 years tenor to see market expectation of inflation in the short term. Five- and Ten-year indicators are highly useful (I believe Larry Summers muscled Treasury into issuing them) but shorter terms are needed, too!
2. BLS needs to do real wage indexes. It does instead detailed unit value indexes of wage payments. The detail minimizes the problem but unit value indices are subject to composition effects. We do not do unit values for good and other services, we should not do them for wages.
3. Treasury (again) should issue a future GDP valued security, a "Trillionth." I would be very useful to know market expectations of GDP. We get some indication of the from stock market movements, but that is far inferior to an actual Trillionth. Issued at the same tenors as TIPS they would lend them selves to deducing expectations of real GDP movements.
There are already 1 year TIPS available in the secondary market. What benefit would an auction have over the current market mechanism?
The very first TIPS auction on January 29 in 1997 was for 10 years, so I doubt Larry Summers had to strong arm anyone into anything. It was the first choice.
Five year TIPS were introduced a few months later. They were the second tenor. Again, seems doubtful any strong arming was required since it was the second choice.
30 year notes weren't introduced until the following year.
Given the pace at which Treasury does things, I imagine TIPS were in progress for a few years before January 1997, when Summers would have been an even more marginal figure.
Been doing one or more of the three since 1987. Not especially good myself, but I was trained by some of the best and it gave me opportunities I had only dreamed of and a body of work to look back on with pride.
I don't think Kolko is making this mistake, but many commenters seem to, so I will correct it. The need for the kind work described here is not in opposition to the kind of work that often gets published and is less directly useful for government. In fact, literature review and modeling are both reliant on prior literature. The problem is not that novel work that requires a PhD to appreciate isn't valuable, it's that we also need to value the work described here.
Yes, it's just about impossible to publish anything relevant to policy in a top journal.
I have nothing to add, but I appreciated this post because it was very "real world" - exactly what Slow Boring should be all about.
How about some research on how bad our system is for getting national data on things like crime or medical statistics. Ideally with some ways to fix it.
I appreciate the insights on what types of research are useful to government agencies for making policy, but there seems to be an implicit assumption, intended or not, of "it should be someone else's responsibility to make sure this research gets funded, completed, and published in a timely fashion." I get it's not easy to change how an institution like the Department of Commerce operates, but... no. If you need additional info to do your job well, then it is also your job to make sure that there is some sort of RFP with a timely process for getting the project scope, funded, and done to produce that info. That may very well mean funding academics, it may not, but either way it's not their responsibility to change how they operate based on your needs unless you make it worth their while. I know, I know, "Fix how government buys things it needs" is a quick way to get bogged down in an endless quagmire, but that doesn't make it less true.
Very interesting to hear the inside scoop on how the sausages are made! The Journal of Economic Literature is indeed a godsend. Are you aware of the Stanford Impact Labs? The intention is precisely to fill this need for research that is boring by journal standards, but necessary and insightful for policy-makers. I believe the director, Jeremy Weinstein, had the same experience as you of having worked in government and found the decision-making to be shockingly unempirical. https://impact.stanford.edu/
I think this issue of bridging between experts (researchers, academics, etc.) and policymakers and policymaking is a very general one. In health care there is a similar disconnect between the available policy research and the use of it by government. I only recall it vauely but I recall an analysis by a med student of all congressional testimony and debate leading up to the demonstration that came to be known as Medicare Health Support. THere was basically no involvment of any health expert, policy research, or even just clinical practitioner in the discussions. The iniative rappidly turned into a give away to "disease management companies" and after hudreds of millions of dollars had been spent disolved into regret, recrimination, and law suits. https://www.nejm.org/doi/full/10.1056/nejmsa1011785 This is what encouraged me to set up and initially fund (using OPM) a program to bring experts (mostly academics) into government for exposure to the policy process https://www.healthandagingpolicy.org/ and policy makers exposure to them. Mostly it seems to have worked well, even though I'm not sure our policy is better yet.
And perhaps worse, those within the policy making sphere don't seem able to accurately evaluate their own efforts or even remember them sometimes. If we aren't going to learn from what we've tried before why do we think we will get better outcomes? As administrations change, as embarassing news is hidden away, and empirical results are overwhelmed by idiology, we keep reinventing the flat tire.
Excellent post!
Great guest. Interesting observation: "Cleaning up contaminated sites for redevelopment creates jobs at a much lower cost per job than job training, which in turn is much more cost-effective than giving businesses tax breaks or grants to create jobs."
I tend to hear this one about retraining diminished by cynicism from the plural of anecdotes: "There's no jobs for the retrained.","The trainees aren't interested in learning, they only want their old job back, they're just there to qualify for the check."
In any case, to the extent this data you say has checked out, I hoped it's not just accounted for and acted upon in federal programs, but widely circulated to states, counties, municipalities, and tribal entities, because the last, less efficient alternative of tax breaks and incentives shaves away fiscal health, and pits localities against each other in a race to the bottom and is an open invitation to self-dealing.
Wait till he sees how useless most academic legal publishing is to practicing lawyers, judges, legislators, or really anyone else except other legal academics.
As a current student staffer on a law journal…legal scholarship is some of the most pointless I’ve ever seen. Legal academia is desperately overdue for some disruption.
As a counterpoint to what I said above, here is an example of academic legal publishing of a type that could be useful to lawmakers (judicial and legislative), because it provides an empirical description of the current law on the ground, in a controversial and contested area where there are competing claims about what and how stable the current law actually is. And where some of those in the debate have an incentive to overstate or misconstrue current law to gain support for legislation to change the law. Note it was not published in a law school law review.
https://patentlyo.com/patent/2023/11/predictability-framework-perspective.html
This is excellent!
I think there is a LOT of low-hanging fruit out there if legal scholars are willing to get more empirical. But they face serious headwinds such as: it’s hard to get data on courts; law tends to draw people from non-technical backgrounds who are uncomfortable with/distrustful of empiricism; student-run journals (and the profession more broadly) value prestigious credentials over content; some legal philosophies explicitly reject empiricism; etc.
Do Supreme Court justices, who say what the Constitution is, care at all what all the brilliant constitutional experts in law schools have to say, other than when it confirms their priors?
Akhil Amar is very frustrated that they don’t listen to him! And at least in Trump v. Anderson he is right!
Just a shout out to two groups the author neglected who do practical econ policy research so well on food and ag: the USDA's Economic Research Service and at the universities, the Extension service.
I don't think he neglected them; I think he was talking particularly from the perspective of Commerce.
Thanks. Yes, I chose my words poorly. Thank you for the correction.
I finished my PhD last year and then moved into government work, and this piece captured a lot of what I think is silly about academic research. At the end of the day the vast majority of it isn’t useful for policy work, no matter how many times they say it has important policy implications in the intro. But I agree that you can’t publish in a top journal or get tenure track doing most of the work you’re describing, unless you’ve already “made it.”
I don't think the main purpose of academic work is to be useful for policy work. It's supposed to be about advancing the field and making discoveries.
Mostly it fails at that too, but that's because in every human endeavor, 80% is crap, 19% is okay, 0.9% is damn good and 0.1% is outstanding.
I didn’t say the purpose of Econ academia is to be useful for policy work. I said much of it claims to be useful for policy work and it isn’t. And I find that aspect of it to be silly. If the field was a little self aware, then I would agree more with what you’re saying.
Academia in general really needs to get fucked. One of the best arguments for abolishing tenure is that then there would be no need to pursue it!
If that’s one of the best arguments then I shudder to think of how shoddy the rest are.
I’m surprised you’re not already aware of them. They’ve been in the news for years.
I'm surprised you're not already aware of all the counterarguments. They've been in the news for years.
I’m not the one wondering what they are, my guy.
“Biden-Harris administration,” really? Is that a thing now? I don’t ever remember reading “Trump-Pence administration, “ or “Obama-Biden”? Kamala has done such a good job protecting our border so I guess she deserves co-billing. https://www.cnn.com/2021/10/15/politics/kamala-harris-border-migration/index.html
Google shows that the phrase “Biden Administration” is 67 times more frequent than “Biden-Harris Administration.” So it’s a little weird, but not unprecedented.
When you work in the administration, it’s respectful and considered good practice to refer to both the President and the VP as the name of the administration. Think you might be reading into this one a bit much.
You must not have been paying attention to the 2020 primary. Biden called it the “Obama-Biden administration all the time; “Biden-Harris” is a continuation of that.
There were, shall we say, strong electoral considerations attending to Biden's gluing himself to Obama during the 2020 campaign.
Yep!
Anything that helps the government get better data is a great idea.
On a related matter, there are economic data that the Government should produce:
1. Treasury should issue TIPS at less than 5 years tenor to see market expectation of inflation in the short term. Five- and Ten-year indicators are highly useful (I believe Larry Summers muscled Treasury into issuing them) but shorter terms are needed, too!
2. BLS needs to do real wage indexes. It does instead detailed unit value indexes of wage payments. The detail minimizes the problem but unit value indices are subject to composition effects. We do not do unit values for good and other services, we should not do them for wages.
3. Treasury (again) should issue a future GDP valued security, a "Trillionth." I would be very useful to know market expectations of GDP. We get some indication of the from stock market movements, but that is far inferior to an actual Trillionth. Issued at the same tenors as TIPS they would lend them selves to deducing expectations of real GDP movements.
Here is my expanded plea:
https://thomaslhutcheson.substack.com/p/improvements-in-macroeconomic-data
There are already 1 year TIPS available in the secondary market. What benefit would an auction have over the current market mechanism?
The very first TIPS auction on January 29 in 1997 was for 10 years, so I doubt Larry Summers had to strong arm anyone into anything. It was the first choice.
Five year TIPS were introduced a few months later. They were the second tenor. Again, seems doubtful any strong arming was required since it was the second choice.
30 year notes weren't introduced until the following year.
Given the pace at which Treasury does things, I imagine TIPS were in progress for a few years before January 1997, when Summers would have been an even more marginal figure.