Vaccines could have been approved in Spring if 2020 if we had challenge trials. Not conducting challenge trials was the single greatest institutional failure during the pandemic.
Two generations ago, our grandfathers stormed beaches, flew combat missions into heavy anti-aircraft fire and steamed across oceans teeming with Uboats to defend our country. Many were conscripted. In 2020, our government would not let healthy young volunteers take a 1 in 2000 chance of death to save tens or hundreds of thousands of people. The risk would be even lower if the vaccines worked!
Our pandemic response stank of cowardice. Early in the pandemic the courts were closed and I didn’t have much to do. I’m not the type to shelter in place, I wanted to help my country. I would have taken a job processing unemployment claims or tracing contacts or volunteered for a challenge trial. There should have been legislation to help people take “pandemic jobs” and then transition back to their regular jobs after the pandemic. I wanted to engage in useful activity, I was willing to take physical risks, and I certainly didn’t want to cower at home and watch people freak out on cable TV. I volunteered for clinical trials and wrote the Russian ambassador asking to participate in the trial of the Sputnik vaccine. Nothing came of my efforts, and I ended up dissipating my energy hiking in the few national forests that weren’t closed. The PPP paid me $20,800 to sit on my ass, so at least my balance sheet didn’t crumble.
We've somehow developed this whole institution called "Medical Ethics" that's managed to become entirely divorced from what people would regard as common sense ethics or what philosophers who study ethics would regard as ethical. There was a letter signed by a who's who of philosophical ethics supporting human challenge trials but it didn't make a difference. https://www.1daysooner.org/us-open-letter
And this is far from the most perverse medical ethics situation I've heard of. In the first medical trials for gene editing which were using an adeoiviral transport to fix subject's Ornithine transcarbamylase deficiency they considered doing the test on babies with a severe form of the disease who would certainly die without this treatment. But the medical ethicists worried that parents would feel they had no choice but to enroll so the only way to conduct the trial ethically would be to give the treatment to subjects with a less severe form of the disease who could probably keep surviving without treatment. But one of the people getting the treatment was old enough to have previously come into contact with the particular adenovirus they chose, had a huge immune response, and died.
ethics. The primary concern is having clear rules that protect practitioners. Eg, give practitioners a “standard of care” that will be a safe harbor even if the patient dies or the client goes to prison. Publix utility is a secondary goal.
"give practitioners a “standard of care” that will be a safe harbor even if the patient dies"
It would be nice if that actually worked. A bad outcome is generally actionable no matter the evidence of deviation from practice- see cerebral palsy cases etc...
I sat on a jury in Boston in the fall of 2007. We awarded 60 odd million to a family with a teenager who had been born with a cerebral palsy. One of the largest such judgments ever given in Massachusetts up to that point. It made the Globe. We found ample evidence of deviation from the "standard of care" the child was due by his medical team.
It’s my experience that people working in fields that have no established professional ethics, as do doctors and lawyers, are very anxious to tell us how self/serving and evil the ethics are. When they are subject to none at all.
I generally agree with you, but I just want to point out that death of Jesse Gelsinger was more complicated than you imply. The researchers involved were pretty careless in their approach and were sanctioned and fined accordingly.
The supply crunch was that they didn't really start working on the supply issue until the vaccine was approved though, wasn't it? There were noises early on about stockpiling before approval but I understand those were defeated by the sort of inflexibility that Matt was talking about in the article.
That doesn't right. Pfizer received EUA in December, and people immediately began receiving jabs. It's true not very large quantities were initially available, so, production had to be increased. What that suggests to me is it takes time (months?) simply for production to even begin. Thus I'm guessing (yes, pure layman's theorizing) that, once the government gave greenlight for production (mid spring 2020?), it took better part of a half year to set up manufacturing. Which meant only modest supplies had been stockpiled by December. But production was fortunately by then on a sharp, upwards trajectory.
My understanding is that the initial doses were from the production lines used in the phase 3? The reporting on the situation I've seen has been very sparse, though. The whole reason I started reading Nightmare Scenario was in the hopes of getting a better picture of what was happening inside Operation Warp Speed but it's focus was elsewhere. I'm really hoping someone comes out with a well reported book directly on the vaccine development/production angle at some point so I can understand exactly what was happening and why better.
Oh, there'll be plenty of books, films and general reportage in the coming years regarding everything related to the pandemic. Of that I have no doubt.
Yep, a new instance of a mature manufacturing process generally takes a month or two to set up and fix the kinks in (plus like 3 months to actually produce mRNA vaccines), and these were about as far from mature manufacturing processes as you can get. I would bet manufacturing started as soon as trials started, but this stuff is hard to make so ramping up takes a long time.
I thought they had stockpiled before approval. Indeed I thought the whole point of both OWS and all the large-scale "at risk" orders from various governments was to encourage exactly that (vaccines that are not authorised yet are still getting millions of doses paid for, e.g. the Sanofi-GSK vaccine, which the French were very keen on in 2020).
I thought it was supposed to be about that to but it didn't end up happening as far as I can tell. The risk orders did speed things up and OWS helped with things like making sure the phase 3 trial was being put together before phase 2 was complete which really sped things up and isn't something a pharma company would normally do so that's an example of flexibility there, though not one that required the FDA to do anything differently if I understand the issue correctly.
That doesn't match what I've read about AstraZeneca production in the UK (and in the EU), but I don't have all the links to hand.
I'm with you on wanting a book or a well-reported longread article about this, especially if it covers all the vaccines (ie including the Chinese and Russian ones) and the Indian production as well as the European and American production of the "western" vaccines.
Disagree. Before trials were done, we didn’t know which vaccines were most effective. There would have been more urgency to ramp up the supply chain if efficacy had been proven early. I don’t think it’s a coincidence that production exploded a couple months after authorization.
We should have been stockpiling the half-dozen or so most credible vaccine candidates. If some weren't as effective we could just write off the few billion dollars loss as tiny compared to the economic damage from the pandemic or even the price of the government response.
My understanding is that at least to some extent we _were_ -- there were a whole bunch of articles about how the Gates foundation was helping pre-build the manufacturing lines for several of the most promising vaccine candidates early on before we knew which ones were going to be efficacious. I'd really love to read some sort of postmortem report on how well that did or did not work.
RNA is very difficult to work with. You look at it the wrong way and it degrades. Scaling up bio pharmaceutical manufacturing can be tough—moving from a 1 liter scale to a 100 liter scale can be a challenge. You also have to train people and that also takes time
We built one factory. Why did we not build 100 in the same time? The lipids are chemicals. The assembly process for the nanoparticles is a pump.
80 years ago Ford made bombers and got one rolling off the line each hour. Yet somehow we can't multiply vaccine production by 10. That's because we didn't even try.
Yeah, my understanding is that the mRNA vaccines in particular were severely bandwidth-limited by shortages in the lipid nanoparticles that they required as an ingredient. But to our host's point, I think that was a reason to be _more_ rather than _less_ flexible about dosing schedules.
While those vaccines had begun production prior to approval, it is also fairly clear that their production ramped up a lot faster after they were approved. If they had been approved months earlier they would have ramped up production earlier.
Challenge trials would be good to establish efficacy rapidly but not effectiveness or safety. I say effectiveness as opposed to efficacy because the volunteers would be chosen for a lack of risk from infection, so you would have little idea how effective the vaccines would be in preventing severe symptoms, hospitalization, or death. At the same time, they would have fewer and less severe adverse reactions than the public, so the risk-benefit picture would be cloudy and lack generalizability. A challenge trial would make sense as a proof of concept, for example, as to whether this newfangled mRNA technology could work but it would be less likely to give all the necessary answers.
At a minimum, challenge trials could greenlight vaccinating healthy young people. this would have been great for essential workers. they would also have justified a massive investment in the supply chain or particular vaccines which were proven to be effective. furthermore, if challenge trials with 30 year olds proved safe, you could do them with 45 year olds, then 60 year olds, etc. each cycle would only take a week or two
The vaccine part of challenge trials with older people could be safe if the vaccine were effective, it's infecting the control group that's problematic
the basic intuition is you don’t need hugely rigorous testing to prove efficacy when efficacy is strong. a sample of 50 is plenty of efficacy is high. you only need the double blinded method and the control in borderline cases where the treatment isn’t that great
you could also have used the diamond princess as a natural control group. put a bunch of vaccinated volunteers on that same ship with a couple infected patients snd let er rip
needing a control group seems to me to be part of the inflexibility. one could use natural experiments with unvaccinated as a control group. the vaccines ended up being effective enough that you didn’t need a very exact estimate of the base rate to know the vaccines were protective
But with a challenge trial, everyone vaccinated is exposed and, assuming less than 100% efficacy, some will get sick. If you have a "natural experiment" control group that is not challenged, it will be largely unexposed (aggressively masked and otherwise avoiding contact) so there will be few people getting sick. You must make assumptions about how many unvaccinated people would have gotten sick if they had been challenged and that level of uncertainty would put big error bars around any estimate of efficacy. A challenge trial as you described would only work if the vaccine were incredibly effective and you can't know that in advance.
I believe that challenge trials would have required a push by the White House itself.
I never understood why Trump didn't try to force challenge trials, as they seem like they would fit with his political style. They seem fairly populist to me.
But what surprised me even more was the fact that we didn't see human challenge trials in another country. Why didn't a country like Argentina offer to do human challenge trials in exchange for getting enough vaccines for their entire population first? Israel did something similar with Pfizer, giving medical data in exchange for early vaccines.
Well now that I now that we have roughly the same annual income, because PPP also paid me $20,800 :) The max that they will pay out a single individual
In NY, the state government reassigned many of its employees to assist with unemployment-related issues, including a few from my office. I cannot speak for everywhere, but the pandemic jobs did at least occur in NY for those who were already state employees. In hindsight, mobilizing civilians who were unemployed due to COVID-related lockdowns would have been more useful than just handing out enhanced unemployment checks, many of which were likely fraudulent.
I certainly don't have any ethical qualms about challenge trials, but I doubt they would have been as useful as you think. When you're testing for the efficacy of a vaccine, you want to know how it would perform in real-world conditions, under exposure to a typical viral load.
That's easy to do when the typical real-world exposure condition is "get bitten by an infected mosquito." That's not so easy when exposure conditions vary as broadly as they do with COVID. You might pick too high a viral load, and then a vaccine that's effective in real-world conditions would appear ineffective. You might pick too low a viral load, and an ineffective vaccine would appear effective.
seems to me a good challenge trial would be to put a vaccinated person in a car with an infected person and have him drive 60 miles or have a vaccinated and infected person eat dinner together. it’s very easy to simulate a natural exposure, you just find an ambulatory, integrated patient
I think this proves my point that we don't know how to do challenge trials rigorously. We could figure that out and have confidence in the results, but would you really want to redesign the car while you're traveling at 80 mph down the highway?
I think "Spring 2020" is more than a bit exaggerated. Maybe September? Also, bureaucracies are not good at changing longstanding practices in mid-stream. As good an idea as challenge trials are, I can easily imagine them screwing them up just because it's not how they're used to doing clinical trials.
Anyone pushing challenge trials now needs to answer the following question: Would you, with the knowledge of hindsight, willingly sign up for a challenge trial for one of the ineffective vaccines? Because there were plenty that didn't make it to market. Merck in particular was one with the weight of many years of vaccine experience that just didn't work. Would you have been willing to take theirs in May 2020? If not, why not?
I didn't sign up for a challenge trial because there weren't any in my country but I did give my name to One Day Sooner as someone to be contacted if they became available. I still have both my kidneys so I'm not you can find an upper bound on how altruistic I am but the incredible reward versus modest risk for volunteering for a challenge made it a no-brainer. I'm not even sure it wasn't selfishly good. Sure there is the physical danger and risk of long term side effects but having my infection be in a controlled environment after an attempted vaccination (90% of phase 3 trials succeed) and being one of the first vaccinated afterwards are all big benefits.
Sorry, 90% of privately funded phase 3 vaccine trials succeed. I don't remember the exact number of government funded ones, its a bit lower but I seem to recall still 80%+? But I was thinking about Moderna and Pfizer when doing research prior to signing up and it's the number relevant to them that I remember.
The numbers are nowhere near that high... of course I can't find them right now but it's under 50% for vaccines. And that number is significantly higher than that for other drugs.
I think you're remembering the numbers for vaccines entering phase 1 or phase 2 rather than those entering phase 3. Lots of vaccines do indeed fail for obvious safety reasons or because they don't cause subjects to generate enough antibodies. But once a vaccine is shown to not cause adverse events in a moderate sized subject group and is also shown to produce decent levels of antibodies in the first two phases then it's pretty rare for more subtle health effects or antibodies not translating into protection to derail it in phase 3.
Ugh -- I see you've already made my point both up-thread and down-thread ... so just view my prior post as one of support. I don't quite get this with-perfect-information-armchair-quarterbacking.
I mean, Merck scrapped two vaccines and that CuraVac one didn't work either. I think the stronger narrative is just how unlikely it was that the mRNA technology worked as well as it did.
There’s a New Yorker cartoon that pops up a lot when economic policy is being discussed: it’s a bunch of disheveled people sitting around a campfire, clearly in some sort of post-apocalyptic scenario, and the person speaking is captioned “but at least for one brief moment we created a lot of value for our shareholders.”
The COVID-era one would need to be re-captioned “but at least for one brief minute we got to fully and rigorously apply our research protocol guidelines and medical ethics training.”
I’m old enough to have cut my teeth, activism-wise, in ACT UP, which had made the entirely correct observation that the FDA was moving at a speed determined by their institutional incentives rather than the actual needs of people dying of a horrific disease. Silly us: in retrospect our one big (if understandable) error was the assumption that things would work any differently if the people dying were “normal”.
For me, the CDC and FDA have proven themselves to be unreliable and untrustworthy at even the most basic public health tasks. Frankly, imo they're a national disgrace.
It's mostly about cloth masks and doesn't even touch on the effectiveness of cloth masks versus surgical, KN95, and N95 masks. AS OF TODAY, THE SITE STILL WARNS PEOPLE NOT TO USE N95 MASKS because they're needed for healthcare workers.
Counter this with Germany, where they mandate medical grade masks and where each household gets free high quality masks provided by the government:
I wonder whether part of the institutional inflexibility is because the public health apparatus is grounded in the scientific process. This isn’t necessarily bad, per se. Of course not! But speaking as a scientist — as a community we collectively learned from experience to never EVER rush, or when we do, to be excruciatingly careful. It’s scientifically risky to rush. You might miss making a key observation. This is basically part of our training. Letting a lengthy scientific process unfold, slowly, collecting tons of data, and refusing to rush, is often what allows the exceptions, the misinterpretations, and the unanticipated/infrequent results to reveal themselves. It’s called the scientific process for a reason; the process is what makes the effort scientific.
So in fighting this institutional inflexibility — an objective I agree with! — it’s probably worth keeping in mind that scientists as a group often don’t want to be rushed, bc rushed science is frequently bad science that can lead us astray, or in the worst cases, hurt people or communities. In my view, there’s a natural conservatism here, inherent to the scientific process, that might be challenging to shift.
This makes a lot of sense, and I think part of the problem is we're asking scientists to be responsible for going beyond science, for issuing recommendations and binding rules rather than just informing that process by publicly saying things like "X we know, Y we are actively debating, Z we can give you informed but uncertain speculation about, A we won't have any insight into for at least years".
One of my hobbyhorses is that making high-stakes decisions of public import, with the force of law behind them, under immense uncertainty, balancing conflicting ethical values, *is properly the role of elected officials*. The public hates elected officials for deeply human reasons – getting elected requires satisfying people with mutually exclusive preferences, making it the path of least resistance for politicians to do weird unnatural stuff – but there really is no substitute for someone accountable to the public and well-versed in their ethical inclinations, compiling the knowledge of experts to arrive at decisions. Scientists, like judges and journalists and soldiers and cops, have been recruited to do this job because people hate politicians, but I think so many institutions would be less compromised and function better if people had more appreciation for the skill and judgment that it takes to mediate between the public's various goals and aspirations and reality.
Yes! Yes! I think your comment and my comment, when put together, create a lot of explanatory power regarding the role of science, public health, and politics during the pandemic.
Exactly. This aspect has been driving me nuts. Scientists have access to data and (hopefully) the power of interpreting it explaining it in a way that decision-makers and the public can basically understand, but the vast majority aren't in a position to sufficiently weigh the *non-scientific* costs and benefits of specific remedies.
You may be right about where the decision-making needs to be centered, the trouble is that our political system seems to select for cowardice and people good at deception. That's not exactly a recipe for success.
Also, we don't elect military generals and they're similarly tasked with making tough life or death calls in an environment of uncertainty. I wonder if there are lessons to be learned from military norms and culture, specifically around the decision-making process and how accountability/blame for bad calls is handled.
The problem is that during a public health emergency, the cost of delaying a decision can be measured in thousands of human lives per day. So a process and institutional culture that's a good fit for normal times can up getting a lot of people killed by inaction.
During an emergency it's all about decision-making under conditions of extreme uncertainty. You make decisions in real-time based on the information you have. Doing nothing while waiting for more data is making a decision just as much as taking an action before the scientific process has worked itself out.
Handling this reality well requires a completely different institutional design and institutional culture than what the FDA and CDC have.
There is that, and in most cases, it's the right thing to do. However the rules covering the grant process have been warped by things like cable TV news complaining that we're spending 50 bucks researching getting fish drunk, which turns out to be the researcher testing various pollutants in the water, like various ethanols.
I keep hearing about R's being anti-science of late, but Millian conservatism ('reactionary, hidebound and stupid') is ever with us. Which has given us decades of attack commercials about spending on research, followed by ever-tightening rules attempting to bulletproof the grant process, which merely slows everything down.
I agree with this, and also would add that MY's comment about the "nimbleness" of very rich people sponsoring some of their favorite researchers really deserves some serious pushback. Biotech is seething with startups whose basic premise is "Hedge Fund Manager's Dad got sick", and by and large they are only really effective at squandering a huge amount of money and resources.
The rich people who are actually effective at driving medical research typically end up building and operating relatively non-nimble bureaucracies with transparency, oversight, etc., in largely the same way that the government does.
You may be totally right here but we really don't know, do we? It would be good to have interviews with people who are currently involved with the process to walk us through it and help us understand either how the institution screwed up or whether, like it or not, it really had to happen this way.
I hate trying to understand people's motives from afar.
The administrators’ careers shouldn’t be in danger here- it should be the politicians who are responsible for changing policy in emergencies that take the heat for mistakes.
This article highlights my daily frustration for the past month or so. There have been several times during this pandemic where the messaging from CDC/public health has sometimes been spin/manipulation of truth, and sometimes rules that are so rigid that they are in direct conflict to what I think is best for my patients. A perfect example was the J&J vaccine, which was clearly not as good as the mRNA ones, but when there was a shortage of vaccines, we needed everyone to get vaccinated with whatever was available. I didn't lie to my patients and tell them it was the same. I said they could always get one of the mRNA vaccines in several months when availability was better. I didn't know this wasn't going to be allowed and someone would have to lie and say they weren't yet vaccinated if they wanted one of the other vaccines.
And even now, immunocompromised people can get a booster with one of the mRNA vaccines, but not the J&J. I have a patient who got J&J about 6 months ago, is now getting chemo for breast cancer and she is not eligible for a booster. I would like her to either get another J&J or complete one of the mRNA series, but she'll have to lie at the pharmacy to do that. A 47 yr-old diabetic patient (mildly overweight) just passed away from a breakthrough Covid infection. His J&J shot was in February. Ok, I know - 1 data point. But this anxiety-producing. I feel hamstrung by rigid rules and regulations that countermand what I think is best for my patients.
There are always situations like that in medicine: recommendations that apply to the population at large, but that may not apply to the individual or off-label recommendations that you make as a doctor given the best data available to you. But for the first time on a large scale, I'm faced with recommendations I would like to make but cannot b/c of inflexible rules.
I have been begging my parents, who got J&J, for two months to lie and get an mRNA shot, but my mom is too nervous about it. Instead, they are just waiting for the feckless FDA to tell them it's okay. I'm not very worried but am frustrated that their risk, even if not that high, is more than it needs to be.
They should do it. Ethics is important to me. Initially, when I heard of people jumping the line to get shots ahead of others, I was outraged and furious. If it's a question of supply, say so. But that's not my impression. What's going on now is arbitrary and harmful. And I'll tell you right now - it's the wild west out there. I've had non-immunocompromised people walk into pharmacies, ask for a booster and get one. I had a patient back in the spring who asked me about getting an mRNA series after having received the J&J the prior month. This was before I knew rules or that this was a fraught conversation. I treated the analysis as a simple risk:benefit calculation for her specifically, stated we didn't have studies or data on mixing vaccines, discussed potential harm (few, theoretical) and potential benefit (small-this was before Delta, but possibly real). She decided to get the series and no one questioned her. It wasn't an issue. She registered for the vaccines and got them. San Francisco public health allowed their J&J residents to get an mRNA series : https://www.cnbc.com/2021/08/03/jj-covid-vaccine-recipients-can-get-supplemental-pfizer-or-moderna-shots-in-san-francisco.html.
See what I mean? Arbitrary and inconsistent.
Let me just say here I'm definitely not necessarily 100% objective here since I'm basing a lot of my recommendations on what I'm seeing personally in the office and LA county. But it is hard to be objective when firm data is scanty as MY's post discussed. My opinion is that your parents should either get a J&J booster or one of the mRNA series if they can. Do it now.
I've always enjoyed this example from Tetlock's book... but it seems the lesson has been overlearned.
"They didn’t need scientific validation. They just knew. Cochrane despised this attitude. He called it “the God complex.”
When hospitals created cardiac care units to treat patients recovering from heart attacks, Cochrane proposed a randomized trial to determine whether the new units delivered better results than the old treatment, which was to send the patient home for monitoring and bed rest. Physicians balked. It was obvious the cardiac care units were superior, they said, and denying patients the best care would be unethical. But Cochrane was not a man to back down. As a prison camp physician treating fellow POWs during World War II he often stood up to the authorities. On occasion, he loudly berated trigger-happy German guards. So Cochrane got his trial: some patients, randomly selected, were sent to the cardiac care units while others were sent home for monitoring and bed rest. Partway through the trial, Cochrane met with a group of the cardiologists who had tried to stop his experiment. He told them that he had preliminary results. The difference in outcomes between the two treatments was not statistically significant, he emphasized, but it appeared that patients might do slightly better in the cardiac care units. “They were vociferous in their abuse: ‘Archie,’ they said, ‘we always thought you were unethical. You must stop the trial at once.’ ” But then Cochrane revealed he had played a little trick. He had reversed the results: home care had done slightly better than the cardiac units. “There was dead silence and I felt rather sick because they were, after all, my medical colleagues.”
This is a point I made in another discussion here, but I think it deserves repeating:
Pfizer submitted its application for Emergency Use Authorization for their vaccine to the FDA the morning of Friday, November 20, 2020. This was not at all a surprise as they had issued many earlier press releases detailing the results of their clinical trials, including one some two weeks earlier announcing that they were beginning the process of submitting the application. The FDA was aware of all this. Presumably they were also aware of how many people were dying every day, though I may be giving them too much credit.
The afternoon of Friday, November 20, 2020, the FDA issued their own press release noting Pfizer's request and announcing that an advisory committee would meet THREE WEEKS LATER on December 10th. That notice said, in part,
"The week of Nov. 23, the FDA intends to issue a Federal Register notice with details of the meeting, which will include information about a public docket for comments. At that time, public comments can be submitted. These comments will be reviewed by the FDA."
Then they took the weekend off. And Thanksgiving and the following weekend.
This is not merely an example of sclerotic and bureaucratic behavior; it is shameful.
The FDA is very much not your friend, and does not deserve your respect. The FDA does not deserve to exist.
Would it be worse than the status quo? I wouldnt go as far as ken, but the FDA needs to get its priorities straight. It should be trying to save lives, not face.
It's not clear what you're alleging. Isn't it possible that the "advisory committee" meets after the science people have had time to digest the data that has been submitted?
That's exactly what happens. The committee doesn't do the work of interpretation or check the validity of the results. It makes a judgment on based upon what the staff (and company) concludes is the proper inference from the data.
You want the FDA to approve the vaccines with a different, faster process. BUT, THAT'S EXACTLY WHAT THEY DID! When they use a different process, they use a different term for it - Emergency Use Authorization. If they approved more quickly with a different process and then claimed that they had done the full process, that would be a lie. So, it's not surprising there's a different term for it.
Maybe we should have used a different term, like Rapid Approval, or something. But, the fact is they did exactly what you're calling for.
And so they had to take a month to schedule a meeting? Their perverse slowness is not excused by the fact that they could have been even more slow - in fact, it’s an indictment of the sluggishness of the regular system.
It's true that the breakdown with EUA isn't _necessarily_ (or only) with the FDA (although "Authorization" sounds more like "We'll let you take it if you really need to")
Maybe if the Military had mandated it that would have mostly pushed towards mandates elsewhere, but that's still an issue with how the government handled things.
Alex Tabarrok has suggested a kind of volunteer system like the military. People could volunteer to try medicines earlier in the process knowing they might not survive. We valorize it in soldiers, police officers and firemen, why not medicine?
Matt has an extremely common, and somewhat bonkers opinion. The FDA is doing precisely what it is supposed to be doing here.
Since April there have been about 200 covid deaths per million in the US. So, if you chose to be unvaccinated for the last five months, then that was roughly your increased chance of death from covid.
The vaccines also have a risk. There have been roughly 35 deaths per million of people who have had the vaccine in that time frame according to VAERS, but the vast majority of those deaths were almost certainly from another source. A covid shot is not going to protect you from an unrelated heart attack.
There has been a total of 3 deaths due to thrombosis with thrombocytopenia syndrome … about 0.02 per million… that are directly attributable to the J&J vaccine. As far as we know, those are the only deaths. But it is not impossible that there are more.
So _on average_, we now know that it is vastly safer to take the vaccine then to be unprotected. The FDA had enough evidence to determine this with Moderna last December, which is why it authorized it for emergency use. And they turned out to be right! The EUA vaccines are vastly safer than not being vaccinated.
But we don’t actually know that the vaccines are safe and effective for all sub-populations. The J&J vaccine, for example, has a higher risk for younger women. The FDA, correctly, decided to briefly halt administration of the J&J vaccine until it could figure out if it was safe for this subpopulation.
Determining that a vaccine is safe and effective on average is extremely difficult. Determining that it is safe and effective for all subpopulations is even more difficult. That’s what the FDA has to figure out before it can give full approval.
Yes, In all likelihood the FDA is going to grind through their data and the bureaucracy will eventually determine that Moderna et al as safe. And then Matthew Yglesias, Tyler Cowen, and everyone on this comment section will crow about being right.
But if the FDA did bow to pressure to become “more flexible”, and they approved the vaccines before being certain, and it turned out that a vaccine caused harm to left-handed Scotsmen (for example), then I doubt any of you would lose too much sleep.
But the FDA would take a massive hit, and the left-handed Scottish antivaxxers would scream vindication, and everyone who received mandatory vaccinations would sue. And then what happens in the next pandemic, when the FDA has already shown it can’t be trusted?
It’s one thing to argue that, as a political question, we should encourage everyone to be vaccinated. But we can’t base our politics on fact, and then use politics to influence the facts.
We know that the vaccines are safe for general use, because we've seen *hundreds of millions of people use them*. Yes, we followed the outcomes of those people with much less specificity than we would for a typical drug trial, but on the other hand there are like thousands of times as many people who have used them as would in a typical drug trial.
There was never going to ever, ever, ever be a drug trial for a generally permitted drug that would suss out whether that drug was going to have a bad effect on sub 0.1% of the population who might use it. Normal FDA policies would not find out if a drug was bad for left-hand Scotsmen.
What the FDA has done is not be flexible about seeing the results of a badly controlled, badly administered trial that none the less has a lot of power because, again, it's got like probably at least 5,000, maybe 50,000 times as many people in it than a typical phase III trial.
The number of people affected by the J&J issue was about 30% of the population, not 0.1%. Even then it took ten million people before there was enough data to spot the issue. Vaccine side effects are really, really rare, and it takes a ton of data and careful analysis to spot issues.
Left handed men with Scotish ancestry make up about .5% of the US population. We may just have enough data to spot an issue with this group, if we thought there might be an issue with them. But it would take trillions of doses for it to show up on a clustering model.
We know the FDA isn’t waiting for arbitrary small subgroup results, because there will never be enough data for that.
But what about African-American men with heart disease? Latino women with high blood pressure? Cancer survivors? We can and should make sure we understand the effects of those subpopulations.
It is important for the FDA to do due diligence to make sure that what they claim is safe and effective, is safe and effective based on what can be reasonably known. They aren’t idiots.
The FDA has a ton of rules. The cliché is that all those rules are written in blood: someone didn’t think to look for something, and people died. Then a lot of people spent a lot of time trying to make sure that it didn’t happen again.
And yeah, the FDA is more worried about deaths from people who choose to take the vaccine then deaths of people who choose not to take the vaccine. What kind of monsters could act differently?
I read this and feel like you missing the entire point that Matt is trying to make.
The FDA and every other government authority has been urging all the left handed Scotsman to get the vaccine! There is no sub population (of adults) that they are suggesting hold off until there is more data!
The real question is - since they are not doing the normal process and they are actually doing an emergency process, why aren't they actually treating it like an EMERGENCY? They are asking almost the entire country to literally derail months/years of their lives, but aren't willing to work around the clock or work through a holiday?
If this is really an emergency that requires shutting down, stay at home, endless masking, etc. then why aren't all the government agencies screaming for funding/flexibility so they can do everything possible to get this over ASAP?
Rules are written in blood, but that doesn't make them good rules, just rules that were written in response to a bad event. "We must do something, this is something, ergo we must do this" is a seductive syllogism but one that I think institutions should try to avoid. "What is the cost, what is the benefit, what is the likely outcome of doing nothing, and how can we do better next time" are the questions that I think we should be trying to ask here, especially when the subject is 4.5M and counting dead humans.
I agree that is a possible failure mode for agencies. There is an argument that that happened with the Nuclear Regulatory Commission, which lead to increasingly costly regulation, which eventually killed the industry. The NRC bore no costs for imposing regulations and building requirements, and massive reputational costs if an accident did happen. Imposing regulations allowed them to push back against constant activist charges of regulatory capture.
I don’t know of any evidence that that dynamic occurs at the FDA. If a drug turns out to have adverse effects, then the pharma company is blamed rather than the FDA. There is no activist community arguing that the FDA is cutting corners.
If there is any regulatory capture, it is by pharma companies that are intensely interested in cutting regulation to reduce their costs. The FDA’s capitulation in aducanumab shows that it needs more internal safeguards, not less.
Note that Matt’s entire argument is about the political effects of the FDA’s decision. The opinions of “Public health twitter” is not evidence.
Classic FDA defense. That’s pretty well the whole story — “there’s a chance doing things differently would hurt the FDA’s recitation!” That it. Abolish the FDA.
People have views on the FDA based on their politics and background. Some people trust the FDA, and some don’t.
The FDA makes decisions on whether pharmaceuticals are safe and effective or not. Sometimes they make the right call (covid vaccines), and sometimes they don’t (aducanumab).
These are completely different questions. The FDA’s reputation for trustworthiness is not the same as the FDA’s actual trustworthiness. It is just insane not to realize that these are different questions.
Where the FDA does fuck up… and it does fuck up (aducanumab)…. Is when it tries to find the middle line between political pressures and the available evidence.
Pharma is hard… Epistemology is hard… Anyone who tells you differently is selling something.
To me the craziest thing is that, while delaying full approval of the vaccines that have been given to millions under the EUA, the FDA went ahead and gave full approval to an Alzheimers drug that did not demonstrate effectiveness in clinical trials, against the advice of every member of their advisory panel for that drug. And yet they had the nerve to talk about “undermining confidence”…
Excellent post. Your point about adherence to protocol as a crutch is well put.The bureaucracy and adherence to protocol can be stifling in healthcare. I should know because Im a nurse who dealt with the worst of the pandemic. I remember that when the pandemic began in March of last year my immediate superior was fretting what to do about covid cause the cdc didn't release any guidelines. I was thinking to just approach the situation like you do with any unknown disease: gown up with an n95 and face shield. Which us what ended up becoming hospital protocol(we are a very well off hospital so we had all the ppe we could want). It shouldn't be difficult to follow best practices without cdc guidance.
But that incident pails in comparison to another healthcare issue: "HIPPA (sic) Violations." The cult of HIPAA is strong in healthcare and is a prime example of willful bureaucratic paralysis in the industry. It is used to prevent collaboration between colleagues all the time. Why sometimes other nurses get angry with me when I check their patients EHR cause I am "violating HIPAA." I'm sorry but this patient you haven't seen in hours has an abnormally high blood pressure and needs it lowered now. Let me look at his chart so I know which meds to give him.
Not saying that HIPAA is unimportant or that patient confidentiality isn't important. Just that it is common in healthcare for HIPAA to be used as a crutch for inaction.
There is other regulatory BS which gets on my nerves. Like how smartwatches can't be used to monitor patient heartrates at my hospital in spite of them performing better than our current monitors. There are just too many headaches in the field with regulations.
This reminds me of all the cop/lawyer/military movies about the guy who “breaks all the rules, but gets things done.” Breaking institutional constraints is a common American concern stretching back to Captain John Smith.
>>>Relatedly, they continue to not recommend that people who got the Johnson & Johnson shot get an mRNA booster even though there is clear data to support that idea. The issue, as Leana Wen of the Milken Institute School of Public Health explains, is that nobody has done a specific safety trial on the mix-and-match<<<
My sense after reading this article is that quite a few of the examples cited by Matt (I've excerpted one, above) aren't issues of lack of flexibility so much as lack of state capacity. Given the gravity of the crisis that confronted the country in the winter of 2020, we should have marshaled much greater resources toward fighting the pandemic. More studies, more "trying everything," more use of available technologies (like smartphone-based contact tracing software), more use of the Defense Production Act. In short, a titanic national effort. And yes, greater capacity would have enabled greater flexibility.
Also, in the realm of public health, I can't help but suspect the country's lack of genuinely universal, government-guaranteed healthcare coverage also plays a role.
American governance as a whole — not just its public health apparatus — isn't operating at a very high level by rich country standards, from what I can tell. I believe the US is suffering a genuine crisis of governance.
I don't totally disagree with this, but did other developed countries do more 'trying everything', contact tracing, etc.? My impression is that a small number of them did use draconian restrictions (South Korea, Taiwan, now Australia)- but, they also seem to have populations that accept that, so it's tough to say that the military locking down whole streets would 'work' in America. In fact, I'm very confident that it wouldn't.
What are the other 'try everything' methods that other developed countries with stronger state capacity did, that the US didn't?
>>I don't totally disagree with this, but did other developed countries do more 'trying everything', contact tracing, etc.?<<
In the main, no, but A) The USA by rights *ought* to have a far greater capacity to do things than other high income countries (4x GDP of our closest competitor, Japan); B) many (certainly not all) of them mounted a much more thorough, serious and unified response to the crisis early on, and so were in a better position to absorb the odd mistake as time went by. America's lack of nimbleness was magnified by some highly egregious failings in the early days, under Trump (incoherent borders policy, badly botched development of covid tests, lack of proper quarantine protocols, widespread civil disobedience — often with the encouragement of government officials — vis-a-vis social distancing measures, and so on.
I believe America's generally chaotic, shambolic and lumbering response to the pandemic from the beginning, under Trump — and the terrible delays and wasted time this caused (we should have been doing MUCH more in January-February-early March) — are the biggest cause of the country's travails; not generalized "lack of flexibility" as such (though to be sure every mistake adds up).
I'm reminded of Vandiver Bush's writing about how the organizational structures required to execute a plan at scale are very different from the structures required to creatively come up with new ideas. In normal times a certain amount of inflexibility/process is a good thing to have in our public health system. But it would be nice to have some means of overriding business as normal in emergencies.
Replacing judgement with process and bureaucracy seldom works. Things change and sometimes it is better to change your priors and hence your actions when you get radically different information.
To defend the FDA, the EUA is flexibility at work: It allowed the manufacturers to distribute the vaccine. There was a recommendation on how to vaccinate based upon the clinical trials, but the FDA has no authority over how health care personnel practice. Health care personnel can make whatever decision they see fit subject to licensure boards, malpractice suits, damage to reputation, etc. Any bright ideas about vaccination that aren't covered by the clinical trials falls under the CDC; that's why they have their own vaccine advisory committee: they can suggest mix and match, boosters, etc.
The difference between the EUA and full approval is that approval is permanent: it's a license, effectively a contract between the manufacturer and FDA that makes the manufacturer accountable for thousands of details regarding manufacturing, safety monitoring and follow-up studies. As an analogy, say you need somewhere to live. the EUA is equivalent to certifying that a house is fit for habitation. Approval is the equivalent of a purchasing contract with all the details surrounding inspections, contingencies, escrow, etc.
Blaming the FDA here seems misplaced, like making it the scapegoat for a lot of societal problems it has no control over. Most notably: the fact that the leaders of one of the major parties - Trump, DiSantis & Co - have been actively throwing sand in the gears of evidence-based public health messaging.
The FDA can bring the horse to water, which it did with the EUA, but can't make it drink. That takes responsible leadership in the rest of society, and without that it doesn't seem likely that FDA bending it's rules on the evidence required for a standard approval would have made much difference.
I think it's instructive to compare how compromised the FDA was during the Trump Administration (minimally w/good leadership from Hahn and, especially, Gottlieb) to CDC (a shitshow)
Gottlieb, from all I can tell, was genuinely impressive by the standards of any administration. It's not surprising he didn't last the whole Trump administration.
I'm sympathetic to your complaint, but I would like to see your positive proposal.
What does flexibility look like in this context?
You write,
"the reason the Moderna vaccine should have full authorization is that nobody in any position of power or authority in the United States is expressing any doubt about the wisdom of taking the Moderna vaccine."
But that is clearly not a policy that can be put into operation. It's also not literally true: there are people in positions of power and authority, e.g. Congressmen, who are anti-vax. Do you want a majority in Congress to decide on pharmaceutical safety and efficacy? A super-majority? A 5-4 SCOTUS ruling? That's obviously the wrong road to go down (as your mention of Trump makes clear).
Instead, I assume you have in mind a principle like this:
As the stakes go up, we should be willing to tolerate greater uncertainty in the approval process.
Currently, the FDA requires Phase III trials enrolling X number of people, showing effect sizes of Y magnitude, with confidence intervals smaller than E.
On your proposal, I take it, we'd have a rule like the following:
when faced with a medical catastrophe that threatens 0.1 million Americans, we reduce X and Y by 10%, and increase E by 10%. If it threatens 0.5 million Americans, then we reduce X and Y by 30%, and increase E by 30%.
That's not great, but it's at least something that can be implemented.
With respect, you're evincing the mindset that is part of the problem. You can't solve this with a new rule because rules can't save you in exigent circumstances. You have to decide: is the desired outcome that we follow all the rules? Or are the rules meant to facilitate good outcomes, and if they are doing the opposite, then we should ignore the rules?
Like, the emergency use authorization was, by definition, stepping outside the normal process, right? The FDA could simply have said, "Using the same criteria we used to grant the emergency use authorization, and in light of the mountains of preliminary data we already have, we are granting full authorization to the vaccine. We will be rigorously following up with monitoring." Similarly, the FDA could have said, "Normally we would wait for the application for emergency use and then invite public comment, but in this case we're going to invite public comment now so that the advisory committee can meet as soon as the application is received." If that breaks the APA or whatever, then fine: agencies get sued constantly, and in the meantime lives are saved. Or how about, "Until we are finished reviewing all this data, FDA employees doing the reviewing are asked to work nights and weekends and we will be keeping the offices open." OPM would have a fit, I'm sure, but do you really think that there would be grumbling?
Government agencies bend, break, or ignore rules all the time. DACA went well beyond discretion in enforcement; it conjured up a whole new administrative procedure out of whole cloth! Pres. Biden just extended an eviction moratorium that he said was probably illegal! It's not like we don't understand this as a culture; for better or worse, we valorize the good guy who is willing to sacrifice his/her career by breaking the rules to get better outcomes.
Obviously, you don't want the FDA to wing it in normal times, but that's the point: procedures, cultures, and mindsets that are adaptive in emergencies are perverse in normal circumstances, and vice versa.
Agreed, though your examples of DACA and the eviction moratorium extension or poorly chosen. As they both were not emergency situations and wrongly put into place.
Unlike initial pandemic response when we didn't know what we were dealing with, nor did we have vaccines.
I dunno. He gave examples of other countries, like Canada doing better, and proposed solutions like redirecting NIH funding. Maybe you need an act of Congress, but Congress seems keen on covid relief.
"He gave examples of other countries, like Canada doing better, and proposed solutions like redirecting NIH funding."
Fair, but the redirection of funding was not a way to speed up approvals. It just would sponsor other research. Which is great, but does not speed up the approval.
Other countries do better on approval? Let's copy them. But some of them are just piggy-backing on the FDA or WHO, so they don't really offer a model.
Vaccines could have been approved in Spring if 2020 if we had challenge trials. Not conducting challenge trials was the single greatest institutional failure during the pandemic.
Two generations ago, our grandfathers stormed beaches, flew combat missions into heavy anti-aircraft fire and steamed across oceans teeming with Uboats to defend our country. Many were conscripted. In 2020, our government would not let healthy young volunteers take a 1 in 2000 chance of death to save tens or hundreds of thousands of people. The risk would be even lower if the vaccines worked!
Our pandemic response stank of cowardice. Early in the pandemic the courts were closed and I didn’t have much to do. I’m not the type to shelter in place, I wanted to help my country. I would have taken a job processing unemployment claims or tracing contacts or volunteered for a challenge trial. There should have been legislation to help people take “pandemic jobs” and then transition back to their regular jobs after the pandemic. I wanted to engage in useful activity, I was willing to take physical risks, and I certainly didn’t want to cower at home and watch people freak out on cable TV. I volunteered for clinical trials and wrote the Russian ambassador asking to participate in the trial of the Sputnik vaccine. Nothing came of my efforts, and I ended up dissipating my energy hiking in the few national forests that weren’t closed. The PPP paid me $20,800 to sit on my ass, so at least my balance sheet didn’t crumble.
We've somehow developed this whole institution called "Medical Ethics" that's managed to become entirely divorced from what people would regard as common sense ethics or what philosophers who study ethics would regard as ethical. There was a letter signed by a who's who of philosophical ethics supporting human challenge trials but it didn't make a difference. https://www.1daysooner.org/us-open-letter
And this is far from the most perverse medical ethics situation I've heard of. In the first medical trials for gene editing which were using an adeoiviral transport to fix subject's Ornithine transcarbamylase deficiency they considered doing the test on babies with a severe form of the disease who would certainly die without this treatment. But the medical ethicists worried that parents would feel they had no choice but to enroll so the only way to conduct the trial ethically would be to give the treatment to subjects with a less severe form of the disease who could probably keep surviving without treatment. But one of the people getting the treatment was old enough to have previously come into contact with the particular adenovirus they chose, had a huge immune response, and died.
Medical ethics is a lot like legal
ethics. The primary concern is having clear rules that protect practitioners. Eg, give practitioners a “standard of care” that will be a safe harbor even if the patient dies or the client goes to prison. Publix utility is a secondary goal.
"give practitioners a “standard of care” that will be a safe harbor even if the patient dies"
It would be nice if that actually worked. A bad outcome is generally actionable no matter the evidence of deviation from practice- see cerebral palsy cases etc...
I sat on a jury in Boston in the fall of 2007. We awarded 60 odd million to a family with a teenager who had been born with a cerebral palsy. One of the largest such judgments ever given in Massachusetts up to that point. It made the Globe. We found ample evidence of deviation from the "standard of care" the child was due by his medical team.
It’s my experience that people working in fields that have no established professional ethics, as do doctors and lawyers, are very anxious to tell us how self/serving and evil the ethics are. When they are subject to none at all.
I am a lawyer
Obviously I had no way of knowing if you are subject to ethics rules or not, which is why I wrote the comment at the level of generality I did.
I generally agree with you, but I just want to point out that death of Jesse Gelsinger was more complicated than you imply. The researchers involved were pretty careless in their approach and were sanctioned and fined accordingly.
To be fair, challenge trials might have brought authorisation forward, but that wouldn't have done much for the supply crunch.
I'm honestly not sure what would have done much for the supply crunch, but pharma certainly seemed to have built up capacity as fast as they could.
The supply crunch was that they didn't really start working on the supply issue until the vaccine was approved though, wasn't it? There were noises early on about stockpiling before approval but I understand those were defeated by the sort of inflexibility that Matt was talking about in the article.
That doesn't right. Pfizer received EUA in December, and people immediately began receiving jabs. It's true not very large quantities were initially available, so, production had to be increased. What that suggests to me is it takes time (months?) simply for production to even begin. Thus I'm guessing (yes, pure layman's theorizing) that, once the government gave greenlight for production (mid spring 2020?), it took better part of a half year to set up manufacturing. Which meant only modest supplies had been stockpiled by December. But production was fortunately by then on a sharp, upwards trajectory.
My understanding is that the initial doses were from the production lines used in the phase 3? The reporting on the situation I've seen has been very sparse, though. The whole reason I started reading Nightmare Scenario was in the hopes of getting a better picture of what was happening inside Operation Warp Speed but it's focus was elsewhere. I'm really hoping someone comes out with a well reported book directly on the vaccine development/production angle at some point so I can understand exactly what was happening and why better.
Oh, there'll be plenty of books, films and general reportage in the coming years regarding everything related to the pandemic. Of that I have no doubt.
Yep, a new instance of a mature manufacturing process generally takes a month or two to set up and fix the kinks in (plus like 3 months to actually produce mRNA vaccines), and these were about as far from mature manufacturing processes as you can get. I would bet manufacturing started as soon as trials started, but this stuff is hard to make so ramping up takes a long time.
I thought they had stockpiled before approval. Indeed I thought the whole point of both OWS and all the large-scale "at risk" orders from various governments was to encourage exactly that (vaccines that are not authorised yet are still getting millions of doses paid for, e.g. the Sanofi-GSK vaccine, which the French were very keen on in 2020).
I thought it was supposed to be about that to but it didn't end up happening as far as I can tell. The risk orders did speed things up and OWS helped with things like making sure the phase 3 trial was being put together before phase 2 was complete which really sped things up and isn't something a pharma company would normally do so that's an example of flexibility there, though not one that required the FDA to do anything differently if I understand the issue correctly.
That doesn't match what I've read about AstraZeneca production in the UK (and in the EU), but I don't have all the links to hand.
I'm with you on wanting a book or a well-reported longread article about this, especially if it covers all the vaccines (ie including the Chinese and Russian ones) and the Indian production as well as the European and American production of the "western" vaccines.
No!!!! The government said start producing it now and we’ll pay for it whether it works or not. We didn’t wait for the trails to be successful.
Disagree. Before trials were done, we didn’t know which vaccines were most effective. There would have been more urgency to ramp up the supply chain if efficacy had been proven early. I don’t think it’s a coincidence that production exploded a couple months after authorization.
We should have been stockpiling the half-dozen or so most credible vaccine candidates. If some weren't as effective we could just write off the few billion dollars loss as tiny compared to the economic damage from the pandemic or even the price of the government response.
My understanding is that at least to some extent we _were_ -- there were a whole bunch of articles about how the Gates foundation was helping pre-build the manufacturing lines for several of the most promising vaccine candidates early on before we knew which ones were going to be efficacious. I'd really love to read some sort of postmortem report on how well that did or did not work.
>>>there were a whole bunch of articles about how the Gates foundation was helping pre-build the manufacturing lines<<<
Exactly. But this process takes time. mRNA vaccines aren't floor boards.
RNA is very difficult to work with. You look at it the wrong way and it degrades. Scaling up bio pharmaceutical manufacturing can be tough—moving from a 1 liter scale to a 100 liter scale can be a challenge. You also have to train people and that also takes time
We built one factory. Why did we not build 100 in the same time? The lipids are chemicals. The assembly process for the nanoparticles is a pump.
80 years ago Ford made bombers and got one rolling off the line each hour. Yet somehow we can't multiply vaccine production by 10. That's because we didn't even try.
Yeah, my understanding is that the mRNA vaccines in particular were severely bandwidth-limited by shortages in the lipid nanoparticles that they required as an ingredient. But to our host's point, I think that was a reason to be _more_ rather than _less_ flexible about dosing schedules.
While those vaccines had begun production prior to approval, it is also fairly clear that their production ramped up a lot faster after they were approved. If they had been approved months earlier they would have ramped up production earlier.
Challenge trials would be good to establish efficacy rapidly but not effectiveness or safety. I say effectiveness as opposed to efficacy because the volunteers would be chosen for a lack of risk from infection, so you would have little idea how effective the vaccines would be in preventing severe symptoms, hospitalization, or death. At the same time, they would have fewer and less severe adverse reactions than the public, so the risk-benefit picture would be cloudy and lack generalizability. A challenge trial would make sense as a proof of concept, for example, as to whether this newfangled mRNA technology could work but it would be less likely to give all the necessary answers.
At a minimum, challenge trials could greenlight vaccinating healthy young people. this would have been great for essential workers. they would also have justified a massive investment in the supply chain or particular vaccines which were proven to be effective. furthermore, if challenge trials with 30 year olds proved safe, you could do them with 45 year olds, then 60 year olds, etc. each cycle would only take a week or two
The vaccine part of challenge trials with older people could be safe if the vaccine were effective, it's infecting the control group that's problematic
the basic intuition is you don’t need hugely rigorous testing to prove efficacy when efficacy is strong. a sample of 50 is plenty of efficacy is high. you only need the double blinded method and the control in borderline cases where the treatment isn’t that great
you could also have used the diamond princess as a natural control group. put a bunch of vaccinated volunteers on that same ship with a couple infected patients snd let er rip
needing a control group seems to me to be part of the inflexibility. one could use natural experiments with unvaccinated as a control group. the vaccines ended up being effective enough that you didn’t need a very exact estimate of the base rate to know the vaccines were protective
But with a challenge trial, everyone vaccinated is exposed and, assuming less than 100% efficacy, some will get sick. If you have a "natural experiment" control group that is not challenged, it will be largely unexposed (aggressively masked and otherwise avoiding contact) so there will be few people getting sick. You must make assumptions about how many unvaccinated people would have gotten sick if they had been challenged and that level of uncertainty would put big error bars around any estimate of efficacy. A challenge trial as you described would only work if the vaccine were incredibly effective and you can't know that in advance.
the Diamond Princess was a perfect natural experiment. d dude it happened early enough that few distances or masked before it was too late
I believe that challenge trials would have required a push by the White House itself.
I never understood why Trump didn't try to force challenge trials, as they seem like they would fit with his political style. They seem fairly populist to me.
But what surprised me even more was the fact that we didn't see human challenge trials in another country. Why didn't a country like Argentina offer to do human challenge trials in exchange for getting enough vaccines for their entire population first? Israel did something similar with Pfizer, giving medical data in exchange for early vaccines.
Well now that I now that we have roughly the same annual income, because PPP also paid me $20,800 :) The max that they will pay out a single individual
If that's the max it could be either of you earns much more (or less) than the other.
In NY, the state government reassigned many of its employees to assist with unemployment-related issues, including a few from my office. I cannot speak for everywhere, but the pandemic jobs did at least occur in NY for those who were already state employees. In hindsight, mobilizing civilians who were unemployed due to COVID-related lockdowns would have been more useful than just handing out enhanced unemployment checks, many of which were likely fraudulent.
I certainly don't have any ethical qualms about challenge trials, but I doubt they would have been as useful as you think. When you're testing for the efficacy of a vaccine, you want to know how it would perform in real-world conditions, under exposure to a typical viral load.
That's easy to do when the typical real-world exposure condition is "get bitten by an infected mosquito." That's not so easy when exposure conditions vary as broadly as they do with COVID. You might pick too high a viral load, and then a vaccine that's effective in real-world conditions would appear ineffective. You might pick too low a viral load, and an ineffective vaccine would appear effective.
seems to me a good challenge trial would be to put a vaccinated person in a car with an infected person and have him drive 60 miles or have a vaccinated and infected person eat dinner together. it’s very easy to simulate a natural exposure, you just find an ambulatory, integrated patient
I think this proves my point that we don't know how to do challenge trials rigorously. We could figure that out and have confidence in the results, but would you really want to redesign the car while you're traveling at 80 mph down the highway?
*infected
I think "Spring 2020" is more than a bit exaggerated. Maybe September? Also, bureaucracies are not good at changing longstanding practices in mid-stream. As good an idea as challenge trials are, I can easily imagine them screwing them up just because it's not how they're used to doing clinical trials.
Anyone pushing challenge trials now needs to answer the following question: Would you, with the knowledge of hindsight, willingly sign up for a challenge trial for one of the ineffective vaccines? Because there were plenty that didn't make it to market. Merck in particular was one with the weight of many years of vaccine experience that just didn't work. Would you have been willing to take theirs in May 2020? If not, why not?
I didn't sign up for a challenge trial because there weren't any in my country but I did give my name to One Day Sooner as someone to be contacted if they became available. I still have both my kidneys so I'm not you can find an upper bound on how altruistic I am but the incredible reward versus modest risk for volunteering for a challenge made it a no-brainer. I'm not even sure it wasn't selfishly good. Sure there is the physical danger and risk of long term side effects but having my infection be in a controlled environment after an attempted vaccination (90% of phase 3 trials succeed) and being one of the first vaccinated afterwards are all big benefits.
Sorry, 90% of privately funded phase 3 vaccine trials succeed. I don't remember the exact number of government funded ones, its a bit lower but I seem to recall still 80%+? But I was thinking about Moderna and Pfizer when doing research prior to signing up and it's the number relevant to them that I remember.
The numbers are nowhere near that high... of course I can't find them right now but it's under 50% for vaccines. And that number is significantly higher than that for other drugs.
I think you're remembering the numbers for vaccines entering phase 1 or phase 2 rather than those entering phase 3. Lots of vaccines do indeed fail for obvious safety reasons or because they don't cause subjects to generate enough antibodies. But once a vaccine is shown to not cause adverse events in a moderate sized subject group and is also shown to produce decent levels of antibodies in the first two phases then it's pretty rare for more subtle health effects or antibodies not translating into protection to derail it in phase 3.
Ugh -- I see you've already made my point both up-thread and down-thread ... so just view my prior post as one of support. I don't quite get this with-perfect-information-armchair-quarterbacking.
I mean, Merck scrapped two vaccines and that CuraVac one didn't work either. I think the stronger narrative is just how unlikely it was that the mRNA technology worked as well as it did.
From some quick googling it looks like Merck failed in phase 1 and CureVac was doing a combined phase 2 and phase 3 trial.
NovaMax also, no? If not, what happened to it?
There’s a New Yorker cartoon that pops up a lot when economic policy is being discussed: it’s a bunch of disheveled people sitting around a campfire, clearly in some sort of post-apocalyptic scenario, and the person speaking is captioned “but at least for one brief moment we created a lot of value for our shareholders.”
The COVID-era one would need to be re-captioned “but at least for one brief minute we got to fully and rigorously apply our research protocol guidelines and medical ethics training.”
I’m old enough to have cut my teeth, activism-wise, in ACT UP, which had made the entirely correct observation that the FDA was moving at a speed determined by their institutional incentives rather than the actual needs of people dying of a horrific disease. Silly us: in retrospect our one big (if understandable) error was the assumption that things would work any differently if the people dying were “normal”.
For me, the CDC and FDA have proven themselves to be unreliable and untrustworthy at even the most basic public health tasks. Frankly, imo they're a national disgrace.
Take a look at the CDC's CURRENT mask guidance
https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/about-face-coverings.html
It's mostly about cloth masks and doesn't even touch on the effectiveness of cloth masks versus surgical, KN95, and N95 masks. AS OF TODAY, THE SITE STILL WARNS PEOPLE NOT TO USE N95 MASKS because they're needed for healthcare workers.
Counter this with Germany, where they mandate medical grade masks and where each household gets free high quality masks provided by the government:
https://www.forbes.com/sites/tommybeer/2021/01/20/germany-mandates-medical-grade-masks/?sh=69ca21de2596
https://www.deutschland.de/en/videos/free-ffp-2-masks-for-27-million-german-citizens
An institution that can't get even this most basic public health guidance right is not fit for purpose. Period. Full stop.
Very much agreed. I would also add that the CDC's only focus should be on communicable diseases. They got mission creep and lost focus.
I wonder whether part of the institutional inflexibility is because the public health apparatus is grounded in the scientific process. This isn’t necessarily bad, per se. Of course not! But speaking as a scientist — as a community we collectively learned from experience to never EVER rush, or when we do, to be excruciatingly careful. It’s scientifically risky to rush. You might miss making a key observation. This is basically part of our training. Letting a lengthy scientific process unfold, slowly, collecting tons of data, and refusing to rush, is often what allows the exceptions, the misinterpretations, and the unanticipated/infrequent results to reveal themselves. It’s called the scientific process for a reason; the process is what makes the effort scientific.
So in fighting this institutional inflexibility — an objective I agree with! — it’s probably worth keeping in mind that scientists as a group often don’t want to be rushed, bc rushed science is frequently bad science that can lead us astray, or in the worst cases, hurt people or communities. In my view, there’s a natural conservatism here, inherent to the scientific process, that might be challenging to shift.
This makes a lot of sense, and I think part of the problem is we're asking scientists to be responsible for going beyond science, for issuing recommendations and binding rules rather than just informing that process by publicly saying things like "X we know, Y we are actively debating, Z we can give you informed but uncertain speculation about, A we won't have any insight into for at least years".
One of my hobbyhorses is that making high-stakes decisions of public import, with the force of law behind them, under immense uncertainty, balancing conflicting ethical values, *is properly the role of elected officials*. The public hates elected officials for deeply human reasons – getting elected requires satisfying people with mutually exclusive preferences, making it the path of least resistance for politicians to do weird unnatural stuff – but there really is no substitute for someone accountable to the public and well-versed in their ethical inclinations, compiling the knowledge of experts to arrive at decisions. Scientists, like judges and journalists and soldiers and cops, have been recruited to do this job because people hate politicians, but I think so many institutions would be less compromised and function better if people had more appreciation for the skill and judgment that it takes to mediate between the public's various goals and aspirations and reality.
Yes! Yes! I think your comment and my comment, when put together, create a lot of explanatory power regarding the role of science, public health, and politics during the pandemic.
Exactly. This aspect has been driving me nuts. Scientists have access to data and (hopefully) the power of interpreting it explaining it in a way that decision-makers and the public can basically understand, but the vast majority aren't in a position to sufficiently weigh the *non-scientific* costs and benefits of specific remedies.
You may be right about where the decision-making needs to be centered, the trouble is that our political system seems to select for cowardice and people good at deception. That's not exactly a recipe for success.
Also, we don't elect military generals and they're similarly tasked with making tough life or death calls in an environment of uncertainty. I wonder if there are lessons to be learned from military norms and culture, specifically around the decision-making process and how accountability/blame for bad calls is handled.
This seems right to me.
The problem is that during a public health emergency, the cost of delaying a decision can be measured in thousands of human lives per day. So a process and institutional culture that's a good fit for normal times can up getting a lot of people killed by inaction.
During an emergency it's all about decision-making under conditions of extreme uncertainty. You make decisions in real-time based on the information you have. Doing nothing while waiting for more data is making a decision just as much as taking an action before the scientific process has worked itself out.
Handling this reality well requires a completely different institutional design and institutional culture than what the FDA and CDC have.
There is that, and in most cases, it's the right thing to do. However the rules covering the grant process have been warped by things like cable TV news complaining that we're spending 50 bucks researching getting fish drunk, which turns out to be the researcher testing various pollutants in the water, like various ethanols.
I keep hearing about R's being anti-science of late, but Millian conservatism ('reactionary, hidebound and stupid') is ever with us. Which has given us decades of attack commercials about spending on research, followed by ever-tightening rules attempting to bulletproof the grant process, which merely slows everything down.
elm
usg has been all kludged up
I agree with this, and also would add that MY's comment about the "nimbleness" of very rich people sponsoring some of their favorite researchers really deserves some serious pushback. Biotech is seething with startups whose basic premise is "Hedge Fund Manager's Dad got sick", and by and large they are only really effective at squandering a huge amount of money and resources.
The rich people who are actually effective at driving medical research typically end up building and operating relatively non-nimble bureaucracies with transparency, oversight, etc., in largely the same way that the government does.
You're right, but I think the delays linked to the bureaucratic drive to minimize career risks of administrators are much more significant.
You may be totally right here but we really don't know, do we? It would be good to have interviews with people who are currently involved with the process to walk us through it and help us understand either how the institution screwed up or whether, like it or not, it really had to happen this way.
I hate trying to understand people's motives from afar.
The administrators’ careers shouldn’t be in danger here- it should be the politicians who are responsible for changing policy in emergencies that take the heat for mistakes.
Not that they do, of course.
Sure, but that doesn't mean we can't rush to get them funding
This article highlights my daily frustration for the past month or so. There have been several times during this pandemic where the messaging from CDC/public health has sometimes been spin/manipulation of truth, and sometimes rules that are so rigid that they are in direct conflict to what I think is best for my patients. A perfect example was the J&J vaccine, which was clearly not as good as the mRNA ones, but when there was a shortage of vaccines, we needed everyone to get vaccinated with whatever was available. I didn't lie to my patients and tell them it was the same. I said they could always get one of the mRNA vaccines in several months when availability was better. I didn't know this wasn't going to be allowed and someone would have to lie and say they weren't yet vaccinated if they wanted one of the other vaccines.
And even now, immunocompromised people can get a booster with one of the mRNA vaccines, but not the J&J. I have a patient who got J&J about 6 months ago, is now getting chemo for breast cancer and she is not eligible for a booster. I would like her to either get another J&J or complete one of the mRNA series, but she'll have to lie at the pharmacy to do that. A 47 yr-old diabetic patient (mildly overweight) just passed away from a breakthrough Covid infection. His J&J shot was in February. Ok, I know - 1 data point. But this anxiety-producing. I feel hamstrung by rigid rules and regulations that countermand what I think is best for my patients.
There are always situations like that in medicine: recommendations that apply to the population at large, but that may not apply to the individual or off-label recommendations that you make as a doctor given the best data available to you. But for the first time on a large scale, I'm faced with recommendations I would like to make but cannot b/c of inflexible rules.
I have been begging my parents, who got J&J, for two months to lie and get an mRNA shot, but my mom is too nervous about it. Instead, they are just waiting for the feckless FDA to tell them it's okay. I'm not very worried but am frustrated that their risk, even if not that high, is more than it needs to be.
They should do it. Ethics is important to me. Initially, when I heard of people jumping the line to get shots ahead of others, I was outraged and furious. If it's a question of supply, say so. But that's not my impression. What's going on now is arbitrary and harmful. And I'll tell you right now - it's the wild west out there. I've had non-immunocompromised people walk into pharmacies, ask for a booster and get one. I had a patient back in the spring who asked me about getting an mRNA series after having received the J&J the prior month. This was before I knew rules or that this was a fraught conversation. I treated the analysis as a simple risk:benefit calculation for her specifically, stated we didn't have studies or data on mixing vaccines, discussed potential harm (few, theoretical) and potential benefit (small-this was before Delta, but possibly real). She decided to get the series and no one questioned her. It wasn't an issue. She registered for the vaccines and got them. San Francisco public health allowed their J&J residents to get an mRNA series : https://www.cnbc.com/2021/08/03/jj-covid-vaccine-recipients-can-get-supplemental-pfizer-or-moderna-shots-in-san-francisco.html.
See what I mean? Arbitrary and inconsistent.
Let me just say here I'm definitely not necessarily 100% objective here since I'm basing a lot of my recommendations on what I'm seeing personally in the office and LA county. But it is hard to be objective when firm data is scanty as MY's post discussed. My opinion is that your parents should either get a J&J booster or one of the mRNA series if they can. Do it now.
I've been nagging my brother over the same issue.
I've always enjoyed this example from Tetlock's book... but it seems the lesson has been overlearned.
"They didn’t need scientific validation. They just knew. Cochrane despised this attitude. He called it “the God complex.”
When hospitals created cardiac care units to treat patients recovering from heart attacks, Cochrane proposed a randomized trial to determine whether the new units delivered better results than the old treatment, which was to send the patient home for monitoring and bed rest. Physicians balked. It was obvious the cardiac care units were superior, they said, and denying patients the best care would be unethical. But Cochrane was not a man to back down. As a prison camp physician treating fellow POWs during World War II he often stood up to the authorities. On occasion, he loudly berated trigger-happy German guards. So Cochrane got his trial: some patients, randomly selected, were sent to the cardiac care units while others were sent home for monitoring and bed rest. Partway through the trial, Cochrane met with a group of the cardiologists who had tried to stop his experiment. He told them that he had preliminary results. The difference in outcomes between the two treatments was not statistically significant, he emphasized, but it appeared that patients might do slightly better in the cardiac care units. “They were vociferous in their abuse: ‘Archie,’ they said, ‘we always thought you were unethical. You must stop the trial at once.’ ” But then Cochrane revealed he had played a little trick. He had reversed the results: home care had done slightly better than the cardiac units. “There was dead silence and I felt rather sick because they were, after all, my medical colleagues.”
Tetlock, Philip E.; Gardner, Dan. Superforecasting (p. 31). Crown. Kindle Edition.
This is a point I made in another discussion here, but I think it deserves repeating:
Pfizer submitted its application for Emergency Use Authorization for their vaccine to the FDA the morning of Friday, November 20, 2020. This was not at all a surprise as they had issued many earlier press releases detailing the results of their clinical trials, including one some two weeks earlier announcing that they were beginning the process of submitting the application. The FDA was aware of all this. Presumably they were also aware of how many people were dying every day, though I may be giving them too much credit.
The afternoon of Friday, November 20, 2020, the FDA issued their own press release noting Pfizer's request and announcing that an advisory committee would meet THREE WEEKS LATER on December 10th. That notice said, in part,
"The week of Nov. 23, the FDA intends to issue a Federal Register notice with details of the meeting, which will include information about a public docket for comments. At that time, public comments can be submitted. These comments will be reviewed by the FDA."
Then they took the weekend off. And Thanksgiving and the following weekend.
This is not merely an example of sclerotic and bureaucratic behavior; it is shameful.
The FDA is very much not your friend, and does not deserve your respect. The FDA does not deserve to exist.
Without the FDA Mike Lindel is on Fox News selling the cure for cancer he cooked up in his bathtub.
He can already do that. Supplements and such are unregulated. Frankly all that seems relatively harmless in comparison to all these debacles.
No - he really can’t. You think the widespread hawking of snake oil would be relatively harmless?
Would it be worse than the status quo? I wouldnt go as far as ken, but the FDA needs to get its priorities straight. It should be trying to save lives, not face.
“We must accept the FDA as currently constituted” does not logically follow from a general desire to protect consumers.
It's not clear what you're alleging. Isn't it possible that the "advisory committee" meets after the science people have had time to digest the data that has been submitted?
That's exactly what happens. The committee doesn't do the work of interpretation or check the validity of the results. It makes a judgment on based upon what the staff (and company) concludes is the proper inference from the data.
Don't forget the FDA slowed down the EUA process by demanding more followup and an expansion. They then did this again with the pediatric trial.
You want the FDA to approve the vaccines with a different, faster process. BUT, THAT'S EXACTLY WHAT THEY DID! When they use a different process, they use a different term for it - Emergency Use Authorization. If they approved more quickly with a different process and then claimed that they had done the full process, that would be a lie. So, it's not surprising there's a different term for it.
Maybe we should have used a different term, like Rapid Approval, or something. But, the fact is they did exactly what you're calling for.
This is completely true, and yet... I don't think it's unreasonable to look at a situation where:
- in retrospect, we had at least three viable vaccine candidates in the lab as early as April 2020
- over 600,000 americans and 4.5 million humans worldwide died (and both numbers still rising)
...and ask "could we have done better?" before patting ourselves on the back for what we did.
And so they had to take a month to schedule a meeting? Their perverse slowness is not excused by the fact that they could have been even more slow - in fact, it’s an indictment of the sluggishness of the regular system.
emergency use isn't approval.
It's true that the breakdown with EUA isn't _necessarily_ (or only) with the FDA (although "Authorization" sounds more like "We'll let you take it if you really need to")
Maybe if the Military had mandated it that would have mostly pushed towards mandates elsewhere, but that's still an issue with how the government handled things.
How long would the clinical trials of a brand new vaccine technology take with you in charge?
Alex Tabarrok has suggested a kind of volunteer system like the military. People could volunteer to try medicines earlier in the process knowing they might not survive. We valorize it in soldiers, police officers and firemen, why not medicine?
Matt has an extremely common, and somewhat bonkers opinion. The FDA is doing precisely what it is supposed to be doing here.
Since April there have been about 200 covid deaths per million in the US. So, if you chose to be unvaccinated for the last five months, then that was roughly your increased chance of death from covid.
The vaccines also have a risk. There have been roughly 35 deaths per million of people who have had the vaccine in that time frame according to VAERS, but the vast majority of those deaths were almost certainly from another source. A covid shot is not going to protect you from an unrelated heart attack.
There has been a total of 3 deaths due to thrombosis with thrombocytopenia syndrome … about 0.02 per million… that are directly attributable to the J&J vaccine. As far as we know, those are the only deaths. But it is not impossible that there are more.
So _on average_, we now know that it is vastly safer to take the vaccine then to be unprotected. The FDA had enough evidence to determine this with Moderna last December, which is why it authorized it for emergency use. And they turned out to be right! The EUA vaccines are vastly safer than not being vaccinated.
But we don’t actually know that the vaccines are safe and effective for all sub-populations. The J&J vaccine, for example, has a higher risk for younger women. The FDA, correctly, decided to briefly halt administration of the J&J vaccine until it could figure out if it was safe for this subpopulation.
Determining that a vaccine is safe and effective on average is extremely difficult. Determining that it is safe and effective for all subpopulations is even more difficult. That’s what the FDA has to figure out before it can give full approval.
Yes, In all likelihood the FDA is going to grind through their data and the bureaucracy will eventually determine that Moderna et al as safe. And then Matthew Yglesias, Tyler Cowen, and everyone on this comment section will crow about being right.
But if the FDA did bow to pressure to become “more flexible”, and they approved the vaccines before being certain, and it turned out that a vaccine caused harm to left-handed Scotsmen (for example), then I doubt any of you would lose too much sleep.
But the FDA would take a massive hit, and the left-handed Scottish antivaxxers would scream vindication, and everyone who received mandatory vaccinations would sue. And then what happens in the next pandemic, when the FDA has already shown it can’t be trusted?
It’s one thing to argue that, as a political question, we should encourage everyone to be vaccinated. But we can’t base our politics on fact, and then use politics to influence the facts.
But this is in fact wrong.
We know that the vaccines are safe for general use, because we've seen *hundreds of millions of people use them*. Yes, we followed the outcomes of those people with much less specificity than we would for a typical drug trial, but on the other hand there are like thousands of times as many people who have used them as would in a typical drug trial.
There was never going to ever, ever, ever be a drug trial for a generally permitted drug that would suss out whether that drug was going to have a bad effect on sub 0.1% of the population who might use it. Normal FDA policies would not find out if a drug was bad for left-hand Scotsmen.
What the FDA has done is not be flexible about seeing the results of a badly controlled, badly administered trial that none the less has a lot of power because, again, it's got like probably at least 5,000, maybe 50,000 times as many people in it than a typical phase III trial.
The number of people affected by the J&J issue was about 30% of the population, not 0.1%. Even then it took ten million people before there was enough data to spot the issue. Vaccine side effects are really, really rare, and it takes a ton of data and careful analysis to spot issues.
Left handed men with Scotish ancestry make up about .5% of the US population. We may just have enough data to spot an issue with this group, if we thought there might be an issue with them. But it would take trillions of doses for it to show up on a clustering model.
We know the FDA isn’t waiting for arbitrary small subgroup results, because there will never be enough data for that.
But what about African-American men with heart disease? Latino women with high blood pressure? Cancer survivors? We can and should make sure we understand the effects of those subpopulations.
It is important for the FDA to do due diligence to make sure that what they claim is safe and effective, is safe and effective based on what can be reasonably known. They aren’t idiots.
The FDA has a ton of rules. The cliché is that all those rules are written in blood: someone didn’t think to look for something, and people died. Then a lot of people spent a lot of time trying to make sure that it didn’t happen again.
And yeah, the FDA is more worried about deaths from people who choose to take the vaccine then deaths of people who choose not to take the vaccine. What kind of monsters could act differently?
I read this and feel like you missing the entire point that Matt is trying to make.
The FDA and every other government authority has been urging all the left handed Scotsman to get the vaccine! There is no sub population (of adults) that they are suggesting hold off until there is more data!
The real question is - since they are not doing the normal process and they are actually doing an emergency process, why aren't they actually treating it like an EMERGENCY? They are asking almost the entire country to literally derail months/years of their lives, but aren't willing to work around the clock or work through a holiday?
If this is really an emergency that requires shutting down, stay at home, endless masking, etc. then why aren't all the government agencies screaming for funding/flexibility so they can do everything possible to get this over ASAP?
Rules are written in blood, but that doesn't make them good rules, just rules that were written in response to a bad event. "We must do something, this is something, ergo we must do this" is a seductive syllogism but one that I think institutions should try to avoid. "What is the cost, what is the benefit, what is the likely outcome of doing nothing, and how can we do better next time" are the questions that I think we should be trying to ask here, especially when the subject is 4.5M and counting dead humans.
I agree that is a possible failure mode for agencies. There is an argument that that happened with the Nuclear Regulatory Commission, which lead to increasingly costly regulation, which eventually killed the industry. The NRC bore no costs for imposing regulations and building requirements, and massive reputational costs if an accident did happen. Imposing regulations allowed them to push back against constant activist charges of regulatory capture.
I don’t know of any evidence that that dynamic occurs at the FDA. If a drug turns out to have adverse effects, then the pharma company is blamed rather than the FDA. There is no activist community arguing that the FDA is cutting corners.
If there is any regulatory capture, it is by pharma companies that are intensely interested in cutting regulation to reduce their costs. The FDA’s capitulation in aducanumab shows that it needs more internal safeguards, not less.
Note that Matt’s entire argument is about the political effects of the FDA’s decision. The opinions of “Public health twitter” is not evidence.
(p.s. Hi Rob! LTNS!)
p.s. Hi N! It is so weird running into people I know in the normcore world. ;)
Classic FDA defense. That’s pretty well the whole story — “there’s a chance doing things differently would hurt the FDA’s recitation!” That it. Abolish the FDA.
This is what I find frustrating.
People have views on the FDA based on their politics and background. Some people trust the FDA, and some don’t.
The FDA makes decisions on whether pharmaceuticals are safe and effective or not. Sometimes they make the right call (covid vaccines), and sometimes they don’t (aducanumab).
These are completely different questions. The FDA’s reputation for trustworthiness is not the same as the FDA’s actual trustworthiness. It is just insane not to realize that these are different questions.
Where the FDA does fuck up… and it does fuck up (aducanumab)…. Is when it tries to find the middle line between political pressures and the available evidence.
Pharma is hard… Epistemology is hard… Anyone who tells you differently is selling something.
Yes, the contrast between institutional inflexibility and the demands for social flexibility have been really striking. This inconsistency was a theme of the Op-ed Peter Singer & I wrote for the Washington Post back in April 2020: https://www.washingtonpost.com/opinions/2020/04/27/pandemic-ethics-case-experiments-human-volunteers/
For anyone blocked by the WaPo paywall, an open access expanded version of our argument is available as a short academic article here: https://journals.sagepub.com/doi/10.1177/1747016120931920
Much more in this vein is available on my blog, for anyone interested:
https://www.philosophyetc.net/2021/01/lessons-from-pandemic.html
To me the craziest thing is that, while delaying full approval of the vaccines that have been given to millions under the EUA, the FDA went ahead and gave full approval to an Alzheimers drug that did not demonstrate effectiveness in clinical trials, against the advice of every member of their advisory panel for that drug. And yet they had the nerve to talk about “undermining confidence”…
Excellent post. Your point about adherence to protocol as a crutch is well put.The bureaucracy and adherence to protocol can be stifling in healthcare. I should know because Im a nurse who dealt with the worst of the pandemic. I remember that when the pandemic began in March of last year my immediate superior was fretting what to do about covid cause the cdc didn't release any guidelines. I was thinking to just approach the situation like you do with any unknown disease: gown up with an n95 and face shield. Which us what ended up becoming hospital protocol(we are a very well off hospital so we had all the ppe we could want). It shouldn't be difficult to follow best practices without cdc guidance.
But that incident pails in comparison to another healthcare issue: "HIPPA (sic) Violations." The cult of HIPAA is strong in healthcare and is a prime example of willful bureaucratic paralysis in the industry. It is used to prevent collaboration between colleagues all the time. Why sometimes other nurses get angry with me when I check their patients EHR cause I am "violating HIPAA." I'm sorry but this patient you haven't seen in hours has an abnormally high blood pressure and needs it lowered now. Let me look at his chart so I know which meds to give him.
Not saying that HIPAA is unimportant or that patient confidentiality isn't important. Just that it is common in healthcare for HIPAA to be used as a crutch for inaction.
There is other regulatory BS which gets on my nerves. Like how smartwatches can't be used to monitor patient heartrates at my hospital in spite of them performing better than our current monitors. There are just too many headaches in the field with regulations.
Yes but they aren’t claiming actual HIPAA provisions. They are claiming HIPAA provision they’ve completely made up.
This reminds me of all the cop/lawyer/military movies about the guy who “breaks all the rules, but gets things done.” Breaking institutional constraints is a common American concern stretching back to Captain John Smith.
>>>Relatedly, they continue to not recommend that people who got the Johnson & Johnson shot get an mRNA booster even though there is clear data to support that idea. The issue, as Leana Wen of the Milken Institute School of Public Health explains, is that nobody has done a specific safety trial on the mix-and-match<<<
My sense after reading this article is that quite a few of the examples cited by Matt (I've excerpted one, above) aren't issues of lack of flexibility so much as lack of state capacity. Given the gravity of the crisis that confronted the country in the winter of 2020, we should have marshaled much greater resources toward fighting the pandemic. More studies, more "trying everything," more use of available technologies (like smartphone-based contact tracing software), more use of the Defense Production Act. In short, a titanic national effort. And yes, greater capacity would have enabled greater flexibility.
Also, in the realm of public health, I can't help but suspect the country's lack of genuinely universal, government-guaranteed healthcare coverage also plays a role.
American governance as a whole — not just its public health apparatus — isn't operating at a very high level by rich country standards, from what I can tell. I believe the US is suffering a genuine crisis of governance.
I don't totally disagree with this, but did other developed countries do more 'trying everything', contact tracing, etc.? My impression is that a small number of them did use draconian restrictions (South Korea, Taiwan, now Australia)- but, they also seem to have populations that accept that, so it's tough to say that the military locking down whole streets would 'work' in America. In fact, I'm very confident that it wouldn't.
What are the other 'try everything' methods that other developed countries with stronger state capacity did, that the US didn't?
>>I don't totally disagree with this, but did other developed countries do more 'trying everything', contact tracing, etc.?<<
In the main, no, but A) The USA by rights *ought* to have a far greater capacity to do things than other high income countries (4x GDP of our closest competitor, Japan); B) many (certainly not all) of them mounted a much more thorough, serious and unified response to the crisis early on, and so were in a better position to absorb the odd mistake as time went by. America's lack of nimbleness was magnified by some highly egregious failings in the early days, under Trump (incoherent borders policy, badly botched development of covid tests, lack of proper quarantine protocols, widespread civil disobedience — often with the encouragement of government officials — vis-a-vis social distancing measures, and so on.
I believe America's generally chaotic, shambolic and lumbering response to the pandemic from the beginning, under Trump — and the terrible delays and wasted time this caused (we should have been doing MUCH more in January-February-early March) — are the biggest cause of the country's travails; not generalized "lack of flexibility" as such (though to be sure every mistake adds up).
I'm reminded of Vandiver Bush's writing about how the organizational structures required to execute a plan at scale are very different from the structures required to creatively come up with new ideas. In normal times a certain amount of inflexibility/process is a good thing to have in our public health system. But it would be nice to have some means of overriding business as normal in emergencies.
Replacing judgement with process and bureaucracy seldom works. Things change and sometimes it is better to change your priors and hence your actions when you get radically different information.
Bureaucracies can't do that with any haste.
To defend the FDA, the EUA is flexibility at work: It allowed the manufacturers to distribute the vaccine. There was a recommendation on how to vaccinate based upon the clinical trials, but the FDA has no authority over how health care personnel practice. Health care personnel can make whatever decision they see fit subject to licensure boards, malpractice suits, damage to reputation, etc. Any bright ideas about vaccination that aren't covered by the clinical trials falls under the CDC; that's why they have their own vaccine advisory committee: they can suggest mix and match, boosters, etc.
The difference between the EUA and full approval is that approval is permanent: it's a license, effectively a contract between the manufacturer and FDA that makes the manufacturer accountable for thousands of details regarding manufacturing, safety monitoring and follow-up studies. As an analogy, say you need somewhere to live. the EUA is equivalent to certifying that a house is fit for habitation. Approval is the equivalent of a purchasing contract with all the details surrounding inspections, contingencies, escrow, etc.
Blaming the FDA here seems misplaced, like making it the scapegoat for a lot of societal problems it has no control over. Most notably: the fact that the leaders of one of the major parties - Trump, DiSantis & Co - have been actively throwing sand in the gears of evidence-based public health messaging.
The FDA can bring the horse to water, which it did with the EUA, but can't make it drink. That takes responsible leadership in the rest of society, and without that it doesn't seem likely that FDA bending it's rules on the evidence required for a standard approval would have made much difference.
I think it's instructive to compare how compromised the FDA was during the Trump Administration (minimally w/good leadership from Hahn and, especially, Gottlieb) to CDC (a shitshow)
Gottlieb, from all I can tell, was genuinely impressive by the standards of any administration. It's not surprising he didn't last the whole Trump administration.
I'm sympathetic to your complaint, but I would like to see your positive proposal.
What does flexibility look like in this context?
You write,
"the reason the Moderna vaccine should have full authorization is that nobody in any position of power or authority in the United States is expressing any doubt about the wisdom of taking the Moderna vaccine."
But that is clearly not a policy that can be put into operation. It's also not literally true: there are people in positions of power and authority, e.g. Congressmen, who are anti-vax. Do you want a majority in Congress to decide on pharmaceutical safety and efficacy? A super-majority? A 5-4 SCOTUS ruling? That's obviously the wrong road to go down (as your mention of Trump makes clear).
Instead, I assume you have in mind a principle like this:
As the stakes go up, we should be willing to tolerate greater uncertainty in the approval process.
Currently, the FDA requires Phase III trials enrolling X number of people, showing effect sizes of Y magnitude, with confidence intervals smaller than E.
On your proposal, I take it, we'd have a rule like the following:
when faced with a medical catastrophe that threatens 0.1 million Americans, we reduce X and Y by 10%, and increase E by 10%. If it threatens 0.5 million Americans, then we reduce X and Y by 30%, and increase E by 30%.
That's not great, but it's at least something that can be implemented.
So what's your positive proposal?
With respect, you're evincing the mindset that is part of the problem. You can't solve this with a new rule because rules can't save you in exigent circumstances. You have to decide: is the desired outcome that we follow all the rules? Or are the rules meant to facilitate good outcomes, and if they are doing the opposite, then we should ignore the rules?
Like, the emergency use authorization was, by definition, stepping outside the normal process, right? The FDA could simply have said, "Using the same criteria we used to grant the emergency use authorization, and in light of the mountains of preliminary data we already have, we are granting full authorization to the vaccine. We will be rigorously following up with monitoring." Similarly, the FDA could have said, "Normally we would wait for the application for emergency use and then invite public comment, but in this case we're going to invite public comment now so that the advisory committee can meet as soon as the application is received." If that breaks the APA or whatever, then fine: agencies get sued constantly, and in the meantime lives are saved. Or how about, "Until we are finished reviewing all this data, FDA employees doing the reviewing are asked to work nights and weekends and we will be keeping the offices open." OPM would have a fit, I'm sure, but do you really think that there would be grumbling?
Government agencies bend, break, or ignore rules all the time. DACA went well beyond discretion in enforcement; it conjured up a whole new administrative procedure out of whole cloth! Pres. Biden just extended an eviction moratorium that he said was probably illegal! It's not like we don't understand this as a culture; for better or worse, we valorize the good guy who is willing to sacrifice his/her career by breaking the rules to get better outcomes.
Obviously, you don't want the FDA to wing it in normal times, but that's the point: procedures, cultures, and mindsets that are adaptive in emergencies are perverse in normal circumstances, and vice versa.
Agreed, though your examples of DACA and the eviction moratorium extension or poorly chosen. As they both were not emergency situations and wrongly put into place.
Unlike initial pandemic response when we didn't know what we were dealing with, nor did we have vaccines.
I dunno. He gave examples of other countries, like Canada doing better, and proposed solutions like redirecting NIH funding. Maybe you need an act of Congress, but Congress seems keen on covid relief.
"He gave examples of other countries, like Canada doing better, and proposed solutions like redirecting NIH funding."
Fair, but the redirection of funding was not a way to speed up approvals. It just would sponsor other research. Which is great, but does not speed up the approval.
Other countries do better on approval? Let's copy them. But some of them are just piggy-backing on the FDA or WHO, so they don't really offer a model.