Again it all seems obvious until you design the experiment. “Just expose people to the virus” does a lot of work for you but it by no means an experimental design. Please interview a scientist on how they would choose to do this experiment and your argument will be stronger. Do you put the test subject in a room with sick people? Inject it? Gas people with it? How large is the exposure? When you do it the standard way, people are exposed to “normal” levels of the virus not some standard that must first be scientifically established. Not all viral infections are the same, quantities and methods matter.
Agree 100%. I'm a biostatistician and the design of the Pfizer/Moderna trials are dead simple, based on decades of clinical experience and statistical research. An undergraduate statistics student could the study design.
For challenge trials, the design is way more complicated. Even determining what results from the trial would mean that it works is extremely hard, because that depends on how contagious the disease is, the exposure setup, and the effectiveness of the vaccine.
The FDA said they would approve any vaccine that is safe and at least 50% effective. I would love for a proponent of challenge trials to describe an experimental design that would assess the effectiveness of a vaccine with a true effectiveness of 50%. It's not a trivial problem!
This is not the attitude that got us to the moon. The specialists should have found a solution to this not-trivial problem. But clearly they felt no responsibility to do so.
Anyone can see that tests were not expedited in the best possible way. It's up to the specialists to figure out what would be a better way. Anyone can identify the problem, only a specialist can pinpoint the ideal solution. Why is that suprising?
Sure, but that takes time. The Pfizer and Moderna trials each lasted about two months after vaccination. Best case scenario, a challenge trial saves a few weeks? A month? Maybe an additional few weeks saved during the enrollment, which would also be smaller/faster.
Obviously, even a month or two faster would save a lot of lives. But any additional time spend researching study design would very quickly eat into those savings. The Pfizer/Moderna trials could be designed in a few hours, so if it takes a month to research and design the challenge trial, it's basically a wash.
I think the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics.
"the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics."
This seems like the place we should all end up converging. Hopefully the people who start thinking about this stuff now will end up coming to some sort of consensus about the ways in which challenge trials will and won't be a good thing in a future pandemic, and can come up with streamlined procedures for expediting the ones that should happen, and clarifying which ones shouldn't happen.
In the mean time, I think there should be a lot of research devoted to ecological challenge trials of respiratory viruses generally. It's only this past year that I finally realized how little we know about the relative role of surfaces, droplets, and aerosols in the transmission of various "cold" viruses and influenza, and about the relative effectiveness of interventions targeted at children or adults, including school closures, social distancing, and mask use.
This is an optimistic vision, but I'm afraid that the consensus you hope for will be forged by hidebound cover-my-ass/"maintain the public trust" FDA types like Eric Topol, Paul Offit, and so on.
Im interviewing to PhD programs right now do you think this is a good gateway to your field or is masters more the standard? Ive been in biomedical research for a while now and I’m wondering if my experience is sufficient already if I decided to go that route. Id like to do basic research regardless just worried about the long game.
If you want to do biomedical research, I'd say a PhD is standard. Quite honestly, a large portion of people who get masters in biostatistics in my department end up in data science jobs in tech.
This seems like a good sort of way to do it, but it seems far from obvious to me that you'd be able to both end up with fewer people infected and a shorter trial time. I guess each round of this trial would take at least a week, to find an infected and infectious person, use them to expose several volunteers, then keep the volunteers isolated until you can be sure they didn't get infected for that interaction. If you start with a large enough number of participants to be sure you'll get significance early, you'll probably be using a larger number of people than the standard Phase 3 trial used (since you're looking at just one afternoon of exposure, rather than the number of exposures that members of the public are averaging); and if you start with smaller populations until you get a sense of the exposure time needed, you'll end up needing to do many rounds (and the standard Phase 3 trials only ended up taking about ten weeks anyway).
These are surmountable study design and methodology questions that come into play once you get past threshold ethical objections to challenge studies at all. At most they come into the ethical discussion only insofar as they might undermine the premise that in a particular case a challenge study is more ethical because it would be faster and thus save lives.
I disagree in that I actually think the experimental design actually drives the ethics. For instance if I have to get a sick patient to cough on my test subject then Im potentially causing another human being to be responsible for the sickness of another who maybe is less qualified to understand the ethics or isn't in a good position to think about the consequences due to illness. I wouldn’t be surprised if there was an ethical way around these concerns but they need to be clearly stated. No amount of hand waving will make this core problem with Matts argument just go away.
That seems like just a routine question of informed consent, which comes up in every study involving human subjects. IRBs are very experienced at handling those. It might be harder to achieve informed consent for some challenge studies but that's not a good reason to categorically take challenge studies off the table.
If I might rephrase: challenge trials are a great way to test the efficacy of a vaccine against deliberate infection, but what we want is a vaccine that works in the real world. And we can't know if the former necessarily implies that latter, particularly with a novel virus.
With all do respect, I am an active, professional, PhD-having research scientist and I don't agree with that. The only way to establish that efficacy from deliberate infection maps directly on to real-world efficacy is to compare the two in rigorous studies, but in the absence of challenge trials, that study is not possible. I think Matt is out of his depths here because running challenge trials *now* would violate fundamental principles of the scientific method; at some point in the future they may become useful, but barring some huge cache of literature comparing the different methods, we're stuck with what we have for now.
Granted, my expertise is (bio)chemistry, so I am not an expert on infectious disease, but there are nuances to the method, frequency and level of exposure for many known viruses that are not strictly multiplicative. I see no reason that would not be true of the novel coronavirus. You would need to design a deliberate infection model that covers a sufficient combination of those (and other) relevant variables to be able to assume that it is sufficiently simulates real-world infection because every downstream assumption would inherent the flaws in the deliberate infection method. You would also need a sufficiently large group of humans to cover the natural variance in human immune responses, co-morbidities, etc.
It is entirely possible that I am entirely wrong, since the devil is usually in the details and I am not an expert. But at first blush, this is simply not how science works.
The way trials are currently designed, the statement that "there was an X% decrease in rates of infection with Y certainty compared to a placebo" is accurate. If you deliberately infect people as part of the trial, the statement narrows to "there was an X% decrease in rates of infection in people deliberately exposed to the virus with Y certainty compared to a placebo". Moreover, both groups would need to be held in controlled conditions for the duration of the study, or you would have to narrow the statement further to include the caveat that you did not control for random exposure.
"The relationship between vaccine efficacy and infection efficacy on success rate is *strictly multiplicative*. A vaccine that reduces prevents 63% of infections against a 70% infection rate is just as efficacious as one that prevents 81% of infections against a 90% infection rate: 90% effective."
I think this is technically correct, but not actually informative. We *define* vaccine efficacy as the ratio of number of infected with the vaccine to number of infected without the vaccine, in two populations with the same other characteristics (including distribution of exposure). But the efficacy is an efficacy *for that population*, and not in general.
There's no particularly good reason to think that a vaccine that has an observed 90% effectiveness in a population that is constantly kissing each other would have the same observed effectiveness in a population that is sharing needles, or a population that is grocery shopping with masks. If the vaccine works by creating a certain level of immune response, it could be that this immune response is 100% effective against viral doses that are below a certain size, but not against doses that are above that size (if they manage to overwhelm the response and infect many cells before the body gets it under control), so that it is much more effective for the grocery store population than for the IV drug population. It could be that the immune response is concentrated in the lymph fluid rather than the bloodstream. It could be that the immune response prevents a mechanism the virus uses to infect blood cells but not lung cells, so that it's more effective for the IV drug users than for the grocery shoppers.
Another way to put this is that in addition to whatever ethical problems they may or may not have, challenge trials have epistemological problems. They give you knowledge about how the vaccine does or doesn't protect against certain styles of lab-based intentional exposure, which doesn't give you much clear sense of whether the vaccine does or doesn't protect against the real-world kinds of exposure people actually get.
However, a friend pointed out to me that once we are considering challenge trials, there's room for them in a *lot* more than just the vaccine side. Right now we are doing all sorts of NPIs (non-pharmaceutical interventions), which are the medical euphemism for things like closing bars, telling people to wear masks and stay outside, and so on, but we have basically zero data as to whether or not these interventions are effective, and if so, how effective.
Once it was clear that this virus was not very harmful to 20-somethings, it seems that it would have been a good idea to do some challenge trials to measure these things. Get a few hundred volunteers to hang out in a bar for science, and do testing first to ensure that most of them tested negative, and a few tested positive, but don't tell them who is who in advance, and see who tested positive afterwards. Then do the same thing, while telling everyone to stay six feet apart, and requiring the bartenders to wear masks. Do the same thing with concerts and grocery stores and meat-packing factories and playgrounds and such.
If we had started this stuff in April, then maybe by September we would have had a better sense of how likely various types of exposure are to spread the virus, and how much different various interventions make to both individual and social risk. It might also have told us which social protocols are particularly effective ones at spreading, which we could then use in vaccine trials a month or two later, rather than relying on unrealistic injection scenarios.
Of course, one thing that understandably raises huge ethical concerns about this is that this is basically the study design of the Tuskegee syphilis experiment - get a bunch of people, test them, don't tell them the results, and then see how the disease progresses and spreads to others when untreated. But if you are working with a known low-risk population, and give everyone information a week after the experiment, and good treatment, and informed consent before the trial, you can mitigate a lot of that.
This seems like a minor technical issue (mainly because if the vaccine works, it will work against any method of exposure, and vice versa). I guess just try to emulate some known superspreader event for each participant?
I think one natural experiment one could do would be to compare the rates of death of healthcare workers to that of the average population. My hypothesis is that they die at higher relative rates than their age cohorts. If so this could demonstrate the way in which heightened viral exposure increases the mortality rate. I suggest this because it would show that a higher environmental dose results in more deaths. The point is we would need to settle on what is a typical dose this all takes time as well. Totally get what you are saying but this is probably how you would have to get at the question of what dosage emulates a super spreader event. I think it’s possible but may be a bit trickier than most imagine.
Further complicating lab infection: there is precedence for the sensitizing effects of a vaccine creating potentially worse outcome in a lab exposure than in wild exposure. The truly informed consent becomes harder because a subject could reasonably expect that the lab infection would mirror a real world infection when it could be much worse not only because of the issues you raise (strain, load, route) but because of the vaccine itself.
This is a great point. Even with COVID19 after all these months, there is considerable uncertainty about the "viral dose" someone receives out in the world and how sick they are likely to get. It seems to be mediated by masks, where masks reduce the dose and thus result in a mild infection. But, there is also quite a bit of variation in how strongly someone's immune system will react to infection. For a truly novel pathogen where the route of infection is not thoroughly understood, it would be difficult to design the challenge trial.
I’m all for challenge studies to get a quick read on efficacy but I don’t see that doing anything about Phase 3 studies. Vaccine Phase 3 studies are not the size they are simply because of efficacy. It’s because you need large trials to identify safety signals that are too rare to pop up in a small Phase 1 trial. Covid is bad but most people haven’t had it and the vast majority of people survive. The vaccine is going to go into *everyone on the planet* in an ideal world. The risk/benefit ratio is hugely critical for a scenario like this, and it’s very challenging to track adverse events in a non-controlled environment. To avoid Ph3 studies you would need a new way to accurately track safety in the general public.
You can separate out the need to do Phase 3s to test efficacy and the need to test safety and just run big safety trials. Here's a proposal along these lines from early in the pandemic related to challenge trials. -- https://www.medrxiv.org/content/10.1101/2020.05.18.20106187v2
This argument is much more logical than the "challenge studies are inherently unethical" response, which is another flavor of the same general story of medical ethicists being totally divorced from the actual study of regular ethics.
Right. We don't talk about this much - maybe because there's the crazier "vaccines cause autism" meme out there - but if you give an intervention to a young population planet-wide, the side effects have to be really close to zero to make it not a problem.
I think this is also a risk we eat no matter what to some level - the youth of today are the longevity test group.
At first I recoiled, but you convinced me on the merits. We have an all volunteer army risking their lives to protect the rest of us and that doesn't seem unethical. At sporting events we could cheer volunteers who risked their lives in challenge trials and offer our gratitude and collective sorrow at funerals for those given placebos. Seems odd, but better than the alternative of greater death.
We also allow people to undergo abdominal surgery and donate one of their kidneys or part of liver. This is a medical procedure with no benefit to the donator that carries some risk of bad outcomes including death. It seems like a very good analogy to me.
>>We let people volunteer to get shot at for the common good, and as bad as COVID may be, for most people it's a lot less risky than getting shot at.
Yes, but... That shouldn't be the crucial point here. Because it's even more crucial that we be willing to use these methods with deadlier diseases. With something like the MEV-1 virus from Contagion, slowing down trials would mean killing millions.
My question: do challenge trials exist in other countries that emphasize the collective over the individual? I am sure an apples-to-apples comparison is challenging considering situations like the suspect trials of Russia’s Sputnik vaccine.
Well, it's a variant of the trolley problem where you get to yell to the person that you're considering switching the track to, "Hey, are you okay with this," and he yells, "Yes!"
Well, no, that person is going to be very very very very likely to be sick and not dead, but there is a very small risk that that person will be dead.
I think the utilitarians are very comfortable with that.
I have a pet theory that humans round all risks up to 100% or down to 0%. Maybe this creates a problem for informed consent? I'd still let the volunteers take the risk, backed by what medical support we can give them.
Saying that people round risks up or down seems appealing for a lot of examples, but it doesn't explain the weird way that people overestimate risks from airplanes and bike riding and underestimate risks from cars and infectious disease. I think familiarity is also said to play a major role in people's estimate of risk, as well as whether the risk is thought to be something that the individual controls or is controlled by someone else.
Totally spot on. I work in aviation more or less, and we get absolutely crazy when non-aviation people get cranky about the risk. The indignation usually goes something like "how can you say that and get in a car??!?!"
Two limitations of the challenge model that challenge trial proponents rarely if ever address.
1. What is needed is a vaccine that prevents infection from real-world exposures to the virus. A challenge trial exposes patients to the virus in a different way, everyone getting a set amount via the same type of exposure. So a challenge trial should be great for clinical proof of concept, measured efficacy in a challenge trial may differ from real-world effectiveness of the same vaccine. Expose people to not enough virus and you may overestimate real-world effectiveness; epose them to too much and you may understimate vaccine potential and even reject an effective vaccine.
2. Challenge trials would be smaller and faster, which means conserably less safety data.
Both Moderna and Pfizer vaccines essentially charged right into Phase 3 without the typically slower process of first showing some efficacy in Phase 2, helped by the government's guaranteed purchase. Either or both of these vaccines could have failed in Phase 3, as many drugs do. Challenge trials would offer the advantage of weeding out ineffective drugs quickly.
I have nothing to add here; I just want to know the most plausible way to make challenge trials happen. I think the best strategy might be persuading the government of Russia or China to do them.
The worry about this is that if Russia or China do something that is judged to be as morally bad as the Tuskegee syphilis experiment, or the Nazi medical trials, then they could set back the whole cause of getting challenge trials approved. You want to be able to show people that it is possible to do challenge trials in a way that appropriately takes the ethical concerns into account, not just show that challenge trials are possible.
I don't think anyone has proposed doing challenge trials on people other than consenting adults. Matt had the patience to write a lot of words but TL;DR: it's frankly bizarre that letting consenting adults put their health at risk for the good of humanity is considered ethically dubious.
The problem with doing it in China, for example, is making sure that all the volunteers actually volunteered. I don't just mean to impugn China here, but also to note that the international press does not presume good will on the behalf of the Chinese and might end up concluding (whether correctly or not) that people were forced to take part.
I lived in China for more than a decade. It may be a police state but it's not a forced-labor camp (okay, maybe parts of Xinjiang). If the international press wants to find out whether subjects in clinical trials are participating of their own free will, they should interview them and see what they say. To make that possible the government would need to be more transparent than it normally prefers, but it's not a heavy lift.
Yes, I agree... part of my point is that to some extent the Western press is biased against China, and that could lead to problems with the Western public perception of trials done in China.
Sigh. No. The main ethical reason for not doing challenge trials is that COVID-19 is a new disease whose long term effects are unknown. Back in March or April, there was essentially no effective treatment; now at least, hospitals have figured out how to reduce the fatality rate somewhat (assuming they're not overrun with patients). But this is not a disease you want to have!
Is the sighing necessary? This “why are you asking such a dumb question” tone has been the dominant one from challenge trial opponents in the discussion I’ve seen and it smacks of “I was taught this was not an option years ago and I’m not confident in my ability to explain why.”
Sorry... the sigh is because this argument has been refuted by others over and over and it's frustrating to have to keep at it.
To be clear, because I guess I haven't been here: I don't think challenge trials are *never* an option, just that (a) the uncertainty around the science with COVID suggests that they would've been a bad idea here, and (b) challenge trial proponents need to be a lot more clear in how the study design lends itself to faster vaccinations. The details matter, a lot.
I admire you for standing for the opposing position and taking the heat from Matt's fanbase but your position here is completely unconvincing to me.
The status quo got alot of people killed or very very sick and has made those of us with high risk loved ones live in constant fear for almost a year now.
IMO public health officials need to do a better job of explaining why allowing all that death and suffering was better than deliberately infecting volunteers, because I find Matt's argument very compelling. I'd be willing to volunteer if it meant my loved ones got a vaccine quickly and would not get sick especially if government gaurenteed good healthcare and death benefits in the event that I got sick or died.
The *status quo* didn't get a lot of people sick; deliberate choices by political leaders to achieve "natural herd immunity" did. Matt did a good job laying out the NPIs and I don't need to discuss the politics of that side of it.
The problem that challenge trial defenders need to answer, as I've stated below, is: what sort of trial design gets you to safe, efficacious doses given to enough people to achieve herd immunity the fastest? Pfizer and Moderna each produced a 95% efficacious vaccine in 10 months! That's three years faster than any previous vaccine. They did it so quickly that manufacturing hasn't been able to keep up with demand, even though they both ramped up manufacturing this summer. So where would the time savings come into play with a challenge trial?
The time savings are abundantly clear. Even if only 3% of the nation had been slowly administered in September/October/November that would have saved a huge number of lives both directly and a lowering of the R0 before the weather got cold.
Every single political argument we are having now about speed/priority is peanuts compared to the huge amount of lives that would have been saved if the system was designed for time-sensitive pandemics.
Sorry, the refutation isn't very convincing. Millions of people have been infected with the coronavirus. Infecting a few thousand on purpose to validate a vaccine seems like a no brainer. And it doesn't need to be 50-50 vaccine and placebo.
I don't buy Lowe's argument. The core of his argument, as you say, was that the virus may have been really bad. If anything, this only boosts the Yglesias angle.
The question boils down to: is our test population the full globe? Or is our test population those volunteers who sign up?
You seem to be saying you'd rather the test population be the full globe because... the virus might be really bad?
I think there's some ethical view which says: we're not responsible for the deaths of 400,000 because those are "natural" whereas we would be directly responsible for the deaths of those 100 volunteers who signed up and got placebo'd and died.
Some people are comparing this to the trolley scenario, but the key distinction is that those 100 dead people are volunteers, who presumably understand the risks. Now the burden falls to you: why are you blocking volunteers from risking their life to save 399,900 people? Are those deaths now "unnatural"? Or are they now directly on the hands of the people who made this miscalculation?
A lot of people in this thread have made a similar argument so I'll try to respond to that here. I'd argue that 400k deaths (in the US alone, not globally) represent a massive policy failure. Or even more explicitly, it was a policy *preference* of those in power (the "let's achieve natural herd immunity" philosophy). I don't think you can argue that a challenge trial would have prevented those deaths unless you can argue that government officials would've been more prone to provide restaurant relief, enforcd mask mandates, increase unemployment insurance, etc. with an earlier vaccine (because they clearly haven't been willing to do so up until now).
More importantly though, the ethical angle looks a lot different in spring/early summer 2020, when such a decision would've been fraught with more unknowns. There simply wasn't the data on the nature of COVID (how to treat it, long term effects, who's more prone to infection, etc.). And again, we know that even people who recover are prone to heart problems and other long term effects. The nature of a challenge trial is philosophically similar to "right to try" advocates for drug trials (which is a larger discussion, but it can be argued that this could lead pharma to turn into the herbal supplements industry). You can tell people that these are the risks, but the picture looks a lot different when it's 3-5 years later (or more) and you're still not fully recovered.
I'd also argue that the "run a challenge trial" proponents need to think carefully about study design. How many people are you running this with? How are you exposing them to the virus? Masked or unmasked? What's the viral load? What's the strain? How long are you exposing them? How are you preventing them from being exposed outside of the confines of the study? How would we measure vaccine safety on a smaller population? A proper challenge trial study would need to answer all these questions and argue that getting to an answer would happen faster than the incredibly fast development we've seen with the Pfizer and Moderna vaccines (which met their targets of a Phase 3 readout in mid-fall 2020).
I think everyone agrees that the actual harm of Trump administration policies is (1) really bad and (2) a total failure for which we could have done better.
But I think the argument for challenge trials isn't that "we need them because we suck at mitigation", it's that the non-challenge trial needs a certain amount of naturally occurring harm to happen to the placebo group in order to demonstrate efficacy. So the harm of the non-challenge trial is either going to happen quickly (because we have crap mitigation) or slowly (because we have good mitigation) but it's going to happen, and the population itself may be suffering harm outside the study in parallel.
The only way around this I see that doesn't involve challenges is to relax the requirement for a double-blind demonstration of efficacy vs placebo by observing that the placebo group becomes ill.
I think I'm not clear on whether your argument is "you can't really have informed consent" or "this is deontologically immoral" (e.g. it's never okay to go infect people on purpose, regardless of how much harm you think you're fixing) or "there are too many unknowns to safely make this harmful action on utilitarian grounds?"
Yes, the studies clearly went faster because of the exploding infection rates. I guess the counterfactual is, in a world where we were mitigating properly, how quickly would it have taken to get a readout? The estimates were 3-4 months longer. So with much lower infection rates, we still would've had approval in early spring 2021, with production already having been ramped up to the point where most of the population could receive their first dose right away. That would be a much better scenario than our current one!
To respond to your question, my feeling is "there are too many unknowns to safely make this harmful action on utilitarian grounds" as you say. I think challenge trials have a place, but not with COVID. Proponents need to argue more clearly how it would've worked because I don't see how lives would've been saved (either in our current world or a world where the mitigation efforts had succeeded more broadly).
Thanks - that's clarifying. Would you say "challenge trials have a place but not with COVID _ever_" or "challenge trials have a place but not with COVID if it were managed non-stupidly?"
In other words (and this is a very, very hypothetical question because if I'm giving you a magic do-over card for how you think things should have gone, why can't you snap your fingers and go "competent mitigation") if it were a sadly fixed and foregone conclusion that we were going to have rampant community spread and hundreds of thousands of casualties, would that (own-goal pandemic) situation be fare game for challenge trials? Or still too much unknown?
I guess I go back to two key points here: (a) the long-term effects of COVID are potentially dangerous enough that we should be very, very careful about exposing people, and (b) I have yet to see a challenge trial design that gets us to fewer deaths/infections than our current state.
Essentially, you're saying "we would have willingly exposed X number of people to COVID in a challenge trial to save Y lives" knowing that 400k US deaths would have happened by now. In such a hypothetical universe, yes you can make that argument. But the particulars matter here. It's an amazing scientific feat that we got 95% efficacious vaccines into people 10 months after COVID was sequenced. Would it have really happened faster a different way?
Were the estimates really only 3 to 4 months longer to gather the data? If everywhere was Australia or New Zealand would it not take years? I think the key point of this entire discussion is that it took only a matter of weeks to develop a vaccine candidate (how smart are those people!) and the rest was just testing. We should be considering every possible approach to speed up testing and having debates about the ethics and science - which this message board is doing. Almost by definition the ethics will never be 'settled'.
Of course with a more definite and shorter timeline there would have been more political will for stronger NPIs.
But even with the same level of NPIs, getting a vaccine a day sooner would've saved thousands of lives. That should have huge moral weight. The ethical issues involved in letting *willing volunteers* get infected with covid are much smaller; maybe dozens of lives rather than thousands.
It's true there were (and are) a lot of unknowns about the effects of covid, but those same unknowns also weigh on the other side of the scale; by delaying a vaccine, you subject thousands of people to those unknown effects as the pandemic plays out naturally.
The questions about the design of the study are real, but it's hard to believe they couldn't have been figured out in a couple days. But we never got past the ethics issues to see.
Getting a *vaccine* a day sooner would have saved thousands of people. But getting *knowledge* of vaccine effectiveness a day sooner wouldn't have saved thousands of people, because vaccine distribution has been hampered more by manufacturing and logistics than by knowledge of effectiveness.
We didn't wait for knowledge of effectiveness to start manufacturing candidates, and we didn't have to wait for knowledge of effectiveness to start hiring injection workers and designing signup websites. It seems likely that instead of 14 million doses manufactured and 5 million given by the end of December, we might have had 2 million manufactured and 1 million given by the end of November, and 15 million manufactured and 8 million given by the end of December. But if we actually continue to ramp up injection capacity, then likely in a few weeks we will reach the same manufacturing bottleneck that we just would have reached sooner.
I'm not sure these issues are as disconnected as you'd hope. For instance, the vaccine rollout was poor even though people had months to prepare. Prep time isn't as good as the real thing. I imagine it's psychologically (and probably legally?) easier to work on mass producing something which is already approved vs. speculative. But I'm not sure how big this effect is. I would love to read a breakdown of the manufacturing timelines and whether they were as fast as possible.
There is also some benefit to actually getting the doses out sooner (reduces more future infections).
This makes total sense to me, but then I've never seen the trolley problem as a difficult problem. You throw the switch, save net lives, sleep like a baby that night. There seems to be this human idea that sins we cause through inaction to be less bad than sins we cause through inaction, which I will never understand.
FWIW, what ethicists traditionally consider to be the "trolley problem" is not "what do you do when you see a switch that can kill one to save five?", but rather "why is it perfectly good to push a switch to kill one to save five, when it would be awful to shove one fat person physically onto the tracks to stop the trolley and save five?" There are many versions of the "kill one to save five" scenario that are discussed in the literature, and most people share the intuitions that in some of them you absolutely should do it, while in others you shouldn't.
(My take is that the examples are badly designed, and people are reacting to the fact that in the real world, the scenarios that people react badly to are ones in which you would be very far from certain that killing one would actually succeed in saving five - of course you shouldn't kill one for a small chance of saving five!)
That's a good point, a key difference between the trolley problem and real life is in the hypothetical scenario you have perfect information and there is no random chance involved.
Under what conditions are challenge trials going to be allowed? In policy, a clear bright line like "under no circumstances should you make volunteers intentionally sick" is valuable. This is effectively Matt's argument with vaccinating by age and the same argument applies with study design.
But the counterfactual world isn't a world where those people DON'T get the virus. In fact, it very deliberately is a world where they DO get the virus, since that's the only way to get the vaccine approved!
So... the argument is we should intentionally infect people in a controlled environment since people are getting it anyway?
I mean, yes, the vaccine readouts came much faster because the infection levels were so high. But that's not really a good argument to do challenge trials! If anything, it's an argument against, since you're not going to be able to ensure that people in such a challenge trial wouldn't be exposed outside of the trial guidelines.
Also, as others have pointed out, you don't get proper safety readouts without a placebo arm. And as we've seen, there are expected side effects. You can't analyze these without a control group for comparison.
Almost. I think the argument is that we should intentionally infect a _small_ number of people because a _large_ number of people are going to get it anyway. If non-challenge trial takes longer, the ethical cost of the non-challenge trial is what goes on in the rest of the community during the long trial if it turns out that the vaccine works (which has a > 0 expected value).
Not exactly... the cost is how long it takes to get people vaccinated. Which as we've seen, depends on manufacturing and distribution (not trivial considerations). Would any company have been able to produce more vaccines right now if they had had their readout 2-3 months earlier? Doesn't seem like that was the rate determining step, seeing as how they all ramped up production well before approval.
Right - you can argue against a consequentialist case for challenge trials on pragmatic grounds, but that argument can actually be settled in a way the philosophical one cannot.
My understanding (from when I volunteered for the challenge trial) is that participants are kept in isolation, ensuring that they won’t expose others and also giving quick access to medical attention.
> I think that such trials are (1) still ethically questionable, (2) fraught with legal complications, (3) are for those reasons and others not likely to run meaningfully faster than regular Phase II trials, and (4) will produce data that may not be able to be extrapolated to the real-world situation.
Putting aside (1) as a question that will depend on your personal ethics, (2) and (3) are both about COVID-19. Matt is proposing to invest in solving those problems in advance of the
*next viral pandemic*. That proposal is entirely consistent with Derek Lowe's points.
(4) is a genuine factual disagreement. Derek Lowe presented an argument, and Matt presented a counter-argument. If you (@J. Willard Gibbs or others) don't think that counter-argument is convincing, why not?
A fair question and thanks for laying it out this way. My feelings on (1) seem to be counter to a lot of the folks here (there's a larger conversation on "right to try" drugs that needs to be had). I think much of the argument in this thread surrounds (2)/(3) but (4) to me is really the crux of it. So I'll state here as I have in other responses -- challenge trials have a place, but they are very, very situation-specific and COVID doesn't fall into that paradigm.
I guess surrounding (2)/(3) -- could we have actually developed something faster than we did using a challenge trial? Because we got these vaccines incredibly fast, to the point where manufacturing is going to be the holdup in getting people dosed and not the trials themselves.
But (4) is the big problem you have to solve. How, exactly, do you design a challenge trial that gets enough doses of a safe, efficacious vaccine into people more quickly than a traditional trial? Because every disease is a little bit different. You can use learnings from a previous vaccine (and with mRNA, we should!) but there's no guarantee that it will be safe or efficacious until you test it.
Traditional phase 3 (non-challenge) trials run the same set of risks, you just don't know what the confounding variables are. What was the weather like during the "real world" trial? What were local infection rates at the time? What was the market price of grapefruit? Do any of these matter? Are either of them divergent from the "real world" conditions that the vaccine will be deployed to the general public in?
The point of a traditional phase 3 non-challenge trial is that by including 30,000 people from a bunch of different sites in different nations in different hemispheres, you can create a study population that approximates the world average in a very large number of characteristics. Importantly, it approximates the world average in terms of the type and duration of exposure - which challenge trials very much would *not* (unless we've already done other trials to determine what sorts of exposure really have been the relevant ones in the world, which is something we absolutely should do, but would likely take longer than a year).
Exposure for the actual phase 3 trials happened under conditions where masks were mandated, many businesses closed, etc etc. It's not the same problem to the same extent, for sure. But I just don't see why the evidential bar has to be so high, for the FDA to just stop banning a medication and let people take it if they want. I think the "right to try" issues are really relevant to the question of the burden of proof here.
I apologize. No pissing match intended. I will try to back off of any rancorous tone I had.
I will not attempt to devise a specific study design. I am not a field expert. But I was under the impression that Lowe opposed Challenge Trials not due to implementation problems, but because he ethically feared exposing participants to a virus with unknown consequences. I’m sure there is an equally interesting and complicated debate to be had about how to properly implement a Challenge Trial but I am not doing so here.
I used the term “In the Wild” to somewhat glibly refer to the Non-Challenge Trial process. My understanding is as such:
- vaccinate some volunteers, call them Group V
- placebo some other volunteers, call them Group P
- let them all run around in the wild for a while
- once some portion of Group P have contracted the virus, this indicates there’s probably been sufficient exposure to Group V
- then check Group V to see if they fared better. If they did? You got yourself a vaccine.
In this sense, in a Non-Challenge trial, placebo participants are still getting exposed to the virus.
The only difference is it is passive exposure rather than active exposure. In a Challenge Trial you _actively_ expose them in a lab, in a Non Challenge Trial you _passively_ expose them on the streets of cities and towns across the world.
This is the first important point: you’re not “saving” any placebo participants from contracting the virus. For any statistically meaningful result (vaccine good? or bad?) some number of people in Group P is getting the virus.
The second important detail is that in the Non Challenge Trial, you require several non-participants, call it Group N, to also get the virus. These are not volunteers. These are ordinary people on the street. It is only through their exposure and transmission that you ever get any results in Group P or Group V!
For any Non-Challenge vaccine to come about, you must involve this non-volunteer Group N! Worse, Group N has to be _massive_ for there to be any statistically meaningful contact with Groups P and V.
When put in these terms, Non-Challenge Trials sound downright vicious! Not only must they involve non-volunteers, they involve _tons_ of them!
Thus, to many of people in this comment thread, they think who but a monster could oppose a Challenge Trial?! (again, assuming you could figure out implementation)
But indeed I don’t think you’re a monster. Nor Lowe. Here is what I think the core of this debate centers around and why many people seem to be talking past each other (and why I think this is such an interesting subject). We have different ethical frameworks and we think differently about active vs passive harm.
It sounds like to some people, the thought of actively harming Group P is worse than the thought of passively allowing harm to come to Group N. At the end of the day, I think this entire debate is that simple.
Personally, I would rather actively harm Group P (assuming they were volunteers) than passively allow harm to come to (a much larger) Group N of non-volunteers. That’s why I like the idea of Challenge Trials.
Let's say (roughly) there were 30,000 people involved in a traditional trial. So 15k in the placebo group, 15k in the vaccine group (this is roughly the breakdown for Moderna/Pfizer/J&J/AZ). To achieve statistical power, the study designers determined that they would check endpoints when 30, then 60, then 90, then 120 people were infected across BOTH groups (again, roughly). Then they would compare the figures of people in who would be infected and measure the efficacy based on the ratio of people infected. So for example, if 80 people were infected in placebo group and 8 in the vaccine group, that's 90% efficacy (again, these were roughly the numbers you saw in Pfizer/Moderna).
Now if you ask me whether you're putting those 80 people at risk -- yes, you are! But now let's say you run a challenge trial and determine you need 10,000 people to achieve statistical power (I'm completely pulling that figure out of thin air... I don't know the actual number you'd need). So you expose 10,000 people to the disease, 1,000 get infected -- 90% efficacy. You've proven the same thing but exposed many more people to the disease. Obviously the numbers are arbitrary here... but are we really going to be able to get a safety readout amongst a cross-section of society with 100 people? Probably not. What about 1,000 people? Unlikely, but let's say for the sake of argument that you can -- that's still 100 people infected and you're greater than your placebo group in the non-challenge trial!
I just think that thinking about this as a trolley problem is getting a little too cute. You need a real world trial design that addresses these issues. It's not enough to say you're harming people passively versus actively -- I just don't see how a challenge trial reduces the harm at all.
I don’t see how you ever can expose fewer people to the virus in non Challenge scenario.
Suppose, as you say, you need ~120 placebo people to have contracted the virus before you’ve got yourself a meaningful sample. At this point you assume ~120 vaccine people also were exposed and so you’re saying you need an exposure group of ~240 people to make any meaningful determination.
You have two ways of exposing these people:
- expose each in an isolated lab (Challenge Trial)
- tell each to walk around NY until you’ve reached that 120 number
Practically by definition, the latter one is going to require that more than 240 people were exposed! By the time you’ve hit your 120 mark, thousands of New Yorkers would needed to have it in order for it to jump to those 240 people.
The numbers factor out. Call 120 N. For any N (short of some weird edge case where N is the size of the planet’s population) you’ll always limit exposure by doing the exposures in a lab rather than doing the exposures in the wild and waiting and hoping.
If you pick an N of 10,000 you will still require less global exposure by exposing each individually than if you sit around and wait for them to be naturally exposed by the rest of the world. Again- because for them to be naturally exposed millions of non-participants will need to get it.
You raise an interesting point about safety vs efficacy that I hadn’t considered. I suppose the goal there is to vaccinate enough people that you understand the dangerous side effects are under some low bar.
I don’t think this presents any categorically new problem in a Challenge vs a Non Challenge:
you can still choose to vaccinate as many as you want/need and you pick some lower subset of the group to expose.
The vaccinated but non-exposed people give you a safety readout. The vaccinated but exposed people give you an efficacy readout. You cross both with a placebo group so you have a baseline for both.
But does any of that change the fact that you’re still at the end of the day being more _efficient_ about exposure (thereby exposing fewer total people)?
Well now you have two arms to your trial, a safety arm and an efficacy arm. And you've complicated matters immensely. Is this going to provide a cleaner statistical readout than either a straight challenge trial or a traditional trial? And is it going to get you to your answer faster?
Again, Pfizer and Moderna both managed to produce safe vaccines that were 95% efficacious in 10 months, which beat the previous record for a vaccine by over three years. They were so fast that they couldn't even ramp up manufacturing to have enough doses ready by the time of approval (pharma thinks about this in great detail, and Pfizer's CEO Albert Bourla was very clear in the summer that they had already started manufacturing at risk). The onus is really, really high for designing a challenge trial that gets you where you need to be faster.
Again it all seems obvious until you design the experiment. “Just expose people to the virus” does a lot of work for you but it by no means an experimental design. Please interview a scientist on how they would choose to do this experiment and your argument will be stronger. Do you put the test subject in a room with sick people? Inject it? Gas people with it? How large is the exposure? When you do it the standard way, people are exposed to “normal” levels of the virus not some standard that must first be scientifically established. Not all viral infections are the same, quantities and methods matter.
Agree 100%. I'm a biostatistician and the design of the Pfizer/Moderna trials are dead simple, based on decades of clinical experience and statistical research. An undergraduate statistics student could the study design.
For challenge trials, the design is way more complicated. Even determining what results from the trial would mean that it works is extremely hard, because that depends on how contagious the disease is, the exposure setup, and the effectiveness of the vaccine.
The FDA said they would approve any vaccine that is safe and at least 50% effective. I would love for a proponent of challenge trials to describe an experimental design that would assess the effectiveness of a vaccine with a true effectiveness of 50%. It's not a trivial problem!
This is not the attitude that got us to the moon. The specialists should have found a solution to this not-trivial problem. But clearly they felt no responsibility to do so.
Anyone can see that tests were not expedited in the best possible way. It's up to the specialists to figure out what would be a better way. Anyone can identify the problem, only a specialist can pinpoint the ideal solution. Why is that suprising?
Sure, but that takes time. The Pfizer and Moderna trials each lasted about two months after vaccination. Best case scenario, a challenge trial saves a few weeks? A month? Maybe an additional few weeks saved during the enrollment, which would also be smaller/faster.
Obviously, even a month or two faster would save a lot of lives. But any additional time spend researching study design would very quickly eat into those savings. The Pfizer/Moderna trials could be designed in a few hours, so if it takes a month to research and design the challenge trial, it's basically a wash.
I think the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics.
"the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics."
This seems like the place we should all end up converging. Hopefully the people who start thinking about this stuff now will end up coming to some sort of consensus about the ways in which challenge trials will and won't be a good thing in a future pandemic, and can come up with streamlined procedures for expediting the ones that should happen, and clarifying which ones shouldn't happen.
In the mean time, I think there should be a lot of research devoted to ecological challenge trials of respiratory viruses generally. It's only this past year that I finally realized how little we know about the relative role of surfaces, droplets, and aerosols in the transmission of various "cold" viruses and influenza, and about the relative effectiveness of interventions targeted at children or adults, including school closures, social distancing, and mask use.
This is an optimistic vision, but I'm afraid that the consensus you hope for will be forged by hidebound cover-my-ass/"maintain the public trust" FDA types like Eric Topol, Paul Offit, and so on.
>>But any additional time spend researching study design would very quickly eat into those savings.
Couldn't they have done that research during the phase 1 and 2 trials?
Im interviewing to PhD programs right now do you think this is a good gateway to your field or is masters more the standard? Ive been in biomedical research for a while now and I’m wondering if my experience is sufficient already if I decided to go that route. Id like to do basic research regardless just worried about the long game.
If you want to do biomedical research, I'd say a PhD is standard. Quite honestly, a large portion of people who get masters in biostatistics in my department end up in data science jobs in tech.
This seems like a good sort of way to do it, but it seems far from obvious to me that you'd be able to both end up with fewer people infected and a shorter trial time. I guess each round of this trial would take at least a week, to find an infected and infectious person, use them to expose several volunteers, then keep the volunteers isolated until you can be sure they didn't get infected for that interaction. If you start with a large enough number of participants to be sure you'll get significance early, you'll probably be using a larger number of people than the standard Phase 3 trial used (since you're looking at just one afternoon of exposure, rather than the number of exposures that members of the public are averaging); and if you start with smaller populations until you get a sense of the exposure time needed, you'll end up needing to do many rounds (and the standard Phase 3 trials only ended up taking about ten weeks anyway).
These are surmountable study design and methodology questions that come into play once you get past threshold ethical objections to challenge studies at all. At most they come into the ethical discussion only insofar as they might undermine the premise that in a particular case a challenge study is more ethical because it would be faster and thus save lives.
I disagree in that I actually think the experimental design actually drives the ethics. For instance if I have to get a sick patient to cough on my test subject then Im potentially causing another human being to be responsible for the sickness of another who maybe is less qualified to understand the ethics or isn't in a good position to think about the consequences due to illness. I wouldn’t be surprised if there was an ethical way around these concerns but they need to be clearly stated. No amount of hand waving will make this core problem with Matts argument just go away.
That seems like just a routine question of informed consent, which comes up in every study involving human subjects. IRBs are very experienced at handling those. It might be harder to achieve informed consent for some challenge studies but that's not a good reason to categorically take challenge studies off the table.
If I might rephrase: challenge trials are a great way to test the efficacy of a vaccine against deliberate infection, but what we want is a vaccine that works in the real world. And we can't know if the former necessarily implies that latter, particularly with a novel virus.
With all do respect, I am an active, professional, PhD-having research scientist and I don't agree with that. The only way to establish that efficacy from deliberate infection maps directly on to real-world efficacy is to compare the two in rigorous studies, but in the absence of challenge trials, that study is not possible. I think Matt is out of his depths here because running challenge trials *now* would violate fundamental principles of the scientific method; at some point in the future they may become useful, but barring some huge cache of literature comparing the different methods, we're stuck with what we have for now.
Granted, my expertise is (bio)chemistry, so I am not an expert on infectious disease, but there are nuances to the method, frequency and level of exposure for many known viruses that are not strictly multiplicative. I see no reason that would not be true of the novel coronavirus. You would need to design a deliberate infection model that covers a sufficient combination of those (and other) relevant variables to be able to assume that it is sufficiently simulates real-world infection because every downstream assumption would inherent the flaws in the deliberate infection method. You would also need a sufficiently large group of humans to cover the natural variance in human immune responses, co-morbidities, etc.
It is entirely possible that I am entirely wrong, since the devil is usually in the details and I am not an expert. But at first blush, this is simply not how science works.
The way trials are currently designed, the statement that "there was an X% decrease in rates of infection with Y certainty compared to a placebo" is accurate. If you deliberately infect people as part of the trial, the statement narrows to "there was an X% decrease in rates of infection in people deliberately exposed to the virus with Y certainty compared to a placebo". Moreover, both groups would need to be held in controlled conditions for the duration of the study, or you would have to narrow the statement further to include the caveat that you did not control for random exposure.
"The relationship between vaccine efficacy and infection efficacy on success rate is *strictly multiplicative*. A vaccine that reduces prevents 63% of infections against a 70% infection rate is just as efficacious as one that prevents 81% of infections against a 90% infection rate: 90% effective."
I think this is technically correct, but not actually informative. We *define* vaccine efficacy as the ratio of number of infected with the vaccine to number of infected without the vaccine, in two populations with the same other characteristics (including distribution of exposure). But the efficacy is an efficacy *for that population*, and not in general.
There's no particularly good reason to think that a vaccine that has an observed 90% effectiveness in a population that is constantly kissing each other would have the same observed effectiveness in a population that is sharing needles, or a population that is grocery shopping with masks. If the vaccine works by creating a certain level of immune response, it could be that this immune response is 100% effective against viral doses that are below a certain size, but not against doses that are above that size (if they manage to overwhelm the response and infect many cells before the body gets it under control), so that it is much more effective for the grocery store population than for the IV drug population. It could be that the immune response is concentrated in the lymph fluid rather than the bloodstream. It could be that the immune response prevents a mechanism the virus uses to infect blood cells but not lung cells, so that it's more effective for the IV drug users than for the grocery shoppers.
Another way to put this is that in addition to whatever ethical problems they may or may not have, challenge trials have epistemological problems. They give you knowledge about how the vaccine does or doesn't protect against certain styles of lab-based intentional exposure, which doesn't give you much clear sense of whether the vaccine does or doesn't protect against the real-world kinds of exposure people actually get.
However, a friend pointed out to me that once we are considering challenge trials, there's room for them in a *lot* more than just the vaccine side. Right now we are doing all sorts of NPIs (non-pharmaceutical interventions), which are the medical euphemism for things like closing bars, telling people to wear masks and stay outside, and so on, but we have basically zero data as to whether or not these interventions are effective, and if so, how effective.
Once it was clear that this virus was not very harmful to 20-somethings, it seems that it would have been a good idea to do some challenge trials to measure these things. Get a few hundred volunteers to hang out in a bar for science, and do testing first to ensure that most of them tested negative, and a few tested positive, but don't tell them who is who in advance, and see who tested positive afterwards. Then do the same thing, while telling everyone to stay six feet apart, and requiring the bartenders to wear masks. Do the same thing with concerts and grocery stores and meat-packing factories and playgrounds and such.
If we had started this stuff in April, then maybe by September we would have had a better sense of how likely various types of exposure are to spread the virus, and how much different various interventions make to both individual and social risk. It might also have told us which social protocols are particularly effective ones at spreading, which we could then use in vaccine trials a month or two later, rather than relying on unrealistic injection scenarios.
Unfortunately, the closest I found was this:
https://www.npr.org/sections/coronavirus-live-updates/2020/08/24/905534790/german-experiment-tests-how-coronavirus-spreads-at-a-concert
Of course, one thing that understandably raises huge ethical concerns about this is that this is basically the study design of the Tuskegee syphilis experiment - get a bunch of people, test them, don't tell them the results, and then see how the disease progresses and spreads to others when untreated. But if you are working with a known low-risk population, and give everyone information a week after the experiment, and good treatment, and informed consent before the trial, you can mitigate a lot of that.
This seems like a minor technical issue (mainly because if the vaccine works, it will work against any method of exposure, and vice versa). I guess just try to emulate some known superspreader event for each participant?
I think one natural experiment one could do would be to compare the rates of death of healthcare workers to that of the average population. My hypothesis is that they die at higher relative rates than their age cohorts. If so this could demonstrate the way in which heightened viral exposure increases the mortality rate. I suggest this because it would show that a higher environmental dose results in more deaths. The point is we would need to settle on what is a typical dose this all takes time as well. Totally get what you are saying but this is probably how you would have to get at the question of what dosage emulates a super spreader event. I think it’s possible but may be a bit trickier than most imagine.
Further complicating lab infection: there is precedence for the sensitizing effects of a vaccine creating potentially worse outcome in a lab exposure than in wild exposure. The truly informed consent becomes harder because a subject could reasonably expect that the lab infection would mirror a real world infection when it could be much worse not only because of the issues you raise (strain, load, route) but because of the vaccine itself.
This is a great point. Even with COVID19 after all these months, there is considerable uncertainty about the "viral dose" someone receives out in the world and how sick they are likely to get. It seems to be mediated by masks, where masks reduce the dose and thus result in a mild infection. But, there is also quite a bit of variation in how strongly someone's immune system will react to infection. For a truly novel pathogen where the route of infection is not thoroughly understood, it would be difficult to design the challenge trial.
I think that trying all the methods in different groups is probably a good idea, but then you get back to needing large study populations.
I’m all for challenge studies to get a quick read on efficacy but I don’t see that doing anything about Phase 3 studies. Vaccine Phase 3 studies are not the size they are simply because of efficacy. It’s because you need large trials to identify safety signals that are too rare to pop up in a small Phase 1 trial. Covid is bad but most people haven’t had it and the vast majority of people survive. The vaccine is going to go into *everyone on the planet* in an ideal world. The risk/benefit ratio is hugely critical for a scenario like this, and it’s very challenging to track adverse events in a non-controlled environment. To avoid Ph3 studies you would need a new way to accurately track safety in the general public.
You can separate out the need to do Phase 3s to test efficacy and the need to test safety and just run big safety trials. Here's a proposal along these lines from early in the pandemic related to challenge trials. -- https://www.medrxiv.org/content/10.1101/2020.05.18.20106187v2
This argument is much more logical than the "challenge studies are inherently unethical" response, which is another flavor of the same general story of medical ethicists being totally divorced from the actual study of regular ethics.
Right. We don't talk about this much - maybe because there's the crazier "vaccines cause autism" meme out there - but if you give an intervention to a young population planet-wide, the side effects have to be really close to zero to make it not a problem.
I think this is also a risk we eat no matter what to some level - the youth of today are the longevity test group.
At first I recoiled, but you convinced me on the merits. We have an all volunteer army risking their lives to protect the rest of us and that doesn't seem unethical. At sporting events we could cheer volunteers who risked their lives in challenge trials and offer our gratitude and collective sorrow at funerals for those given placebos. Seems odd, but better than the alternative of greater death.
We also allow people to undergo abdominal surgery and donate one of their kidneys or part of liver. This is a medical procedure with no benefit to the donator that carries some risk of bad outcomes including death. It seems like a very good analogy to me.
>>We let people volunteer to get shot at for the common good, and as bad as COVID may be, for most people it's a lot less risky than getting shot at.
Yes, but... That shouldn't be the crucial point here. Because it's even more crucial that we be willing to use these methods with deadlier diseases. With something like the MEV-1 virus from Contagion, slowing down trials would mean killing millions.
It’s the trolley problem!
My question: do challenge trials exist in other countries that emphasize the collective over the individual? I am sure an apples-to-apples comparison is challenging considering situations like the suspect trials of Russia’s Sputnik vaccine.
Well, it's a variant of the trolley problem where you get to yell to the person that you're considering switching the track to, "Hey, are you okay with this," and he yells, "Yes!"
Well put!
Well, no, that person is going to be very very very very likely to be sick and not dead, but there is a very small risk that that person will be dead.
I think the utilitarians are very comfortable with that.
I have a pet theory that humans round all risks up to 100% or down to 0%. Maybe this creates a problem for informed consent? I'd still let the volunteers take the risk, backed by what medical support we can give them.
Saying that people round risks up or down seems appealing for a lot of examples, but it doesn't explain the weird way that people overestimate risks from airplanes and bike riding and underestimate risks from cars and infectious disease. I think familiarity is also said to play a major role in people's estimate of risk, as well as whether the risk is thought to be something that the individual controls or is controlled by someone else.
Totally spot on. I work in aviation more or less, and we get absolutely crazy when non-aviation people get cranky about the risk. The indignation usually goes something like "how can you say that and get in a car??!?!"
Right, you have to use inactivated or attenuated trolley-cars.
Two limitations of the challenge model that challenge trial proponents rarely if ever address.
1. What is needed is a vaccine that prevents infection from real-world exposures to the virus. A challenge trial exposes patients to the virus in a different way, everyone getting a set amount via the same type of exposure. So a challenge trial should be great for clinical proof of concept, measured efficacy in a challenge trial may differ from real-world effectiveness of the same vaccine. Expose people to not enough virus and you may overestimate real-world effectiveness; epose them to too much and you may understimate vaccine potential and even reject an effective vaccine.
2. Challenge trials would be smaller and faster, which means conserably less safety data.
Both Moderna and Pfizer vaccines essentially charged right into Phase 3 without the typically slower process of first showing some efficacy in Phase 2, helped by the government's guaranteed purchase. Either or both of these vaccines could have failed in Phase 3, as many drugs do. Challenge trials would offer the advantage of weeding out ineffective drugs quickly.
I have nothing to add here; I just want to know the most plausible way to make challenge trials happen. I think the best strategy might be persuading the government of Russia or China to do them.
The worry about this is that if Russia or China do something that is judged to be as morally bad as the Tuskegee syphilis experiment, or the Nazi medical trials, then they could set back the whole cause of getting challenge trials approved. You want to be able to show people that it is possible to do challenge trials in a way that appropriately takes the ethical concerns into account, not just show that challenge trials are possible.
I don't think anyone has proposed doing challenge trials on people other than consenting adults. Matt had the patience to write a lot of words but TL;DR: it's frankly bizarre that letting consenting adults put their health at risk for the good of humanity is considered ethically dubious.
The problem with doing it in China, for example, is making sure that all the volunteers actually volunteered. I don't just mean to impugn China here, but also to note that the international press does not presume good will on the behalf of the Chinese and might end up concluding (whether correctly or not) that people were forced to take part.
I lived in China for more than a decade. It may be a police state but it's not a forced-labor camp (okay, maybe parts of Xinjiang). If the international press wants to find out whether subjects in clinical trials are participating of their own free will, they should interview them and see what they say. To make that possible the government would need to be more transparent than it normally prefers, but it's not a heavy lift.
Yes, I agree... part of my point is that to some extent the Western press is biased against China, and that could lead to problems with the Western public perception of trials done in China.
Matt, may I arrange to have this piece translated?
Sigh. No. The main ethical reason for not doing challenge trials is that COVID-19 is a new disease whose long term effects are unknown. Back in March or April, there was essentially no effective treatment; now at least, hospitals have figured out how to reduce the fatality rate somewhat (assuming they're not overrun with patients). But this is not a disease you want to have!
Rather than rant for several paragraphs, I'll defer to this Derek Lowe post, who has laid it out far more clearly than I ever could: https://blogs.sciencemag.org/pipeline/archives/2020/07/02/challenge-testing
Is the sighing necessary? This “why are you asking such a dumb question” tone has been the dominant one from challenge trial opponents in the discussion I’ve seen and it smacks of “I was taught this was not an option years ago and I’m not confident in my ability to explain why.”
Sorry... the sigh is because this argument has been refuted by others over and over and it's frustrating to have to keep at it.
To be clear, because I guess I haven't been here: I don't think challenge trials are *never* an option, just that (a) the uncertainty around the science with COVID suggests that they would've been a bad idea here, and (b) challenge trial proponents need to be a lot more clear in how the study design lends itself to faster vaccinations. The details matter, a lot.
It hasn't been "refuted"; the arguments have been aired and they are not convincing to lots of people (including me).
I admire you for standing for the opposing position and taking the heat from Matt's fanbase but your position here is completely unconvincing to me.
The status quo got alot of people killed or very very sick and has made those of us with high risk loved ones live in constant fear for almost a year now.
IMO public health officials need to do a better job of explaining why allowing all that death and suffering was better than deliberately infecting volunteers, because I find Matt's argument very compelling. I'd be willing to volunteer if it meant my loved ones got a vaccine quickly and would not get sick especially if government gaurenteed good healthcare and death benefits in the event that I got sick or died.
The *status quo* didn't get a lot of people sick; deliberate choices by political leaders to achieve "natural herd immunity" did. Matt did a good job laying out the NPIs and I don't need to discuss the politics of that side of it.
The problem that challenge trial defenders need to answer, as I've stated below, is: what sort of trial design gets you to safe, efficacious doses given to enough people to achieve herd immunity the fastest? Pfizer and Moderna each produced a 95% efficacious vaccine in 10 months! That's three years faster than any previous vaccine. They did it so quickly that manufacturing hasn't been able to keep up with demand, even though they both ramped up manufacturing this summer. So where would the time savings come into play with a challenge trial?
The time savings are abundantly clear. Even if only 3% of the nation had been slowly administered in September/October/November that would have saved a huge number of lives both directly and a lowering of the R0 before the weather got cold.
Every single political argument we are having now about speed/priority is peanuts compared to the huge amount of lives that would have been saved if the system was designed for time-sensitive pandemics.
Sorry, the refutation isn't very convincing. Millions of people have been infected with the coronavirus. Infecting a few thousand on purpose to validate a vaccine seems like a no brainer. And it doesn't need to be 50-50 vaccine and placebo.
I don't buy Lowe's argument. The core of his argument, as you say, was that the virus may have been really bad. If anything, this only boosts the Yglesias angle.
The question boils down to: is our test population the full globe? Or is our test population those volunteers who sign up?
You seem to be saying you'd rather the test population be the full globe because... the virus might be really bad?
I think there's some ethical view which says: we're not responsible for the deaths of 400,000 because those are "natural" whereas we would be directly responsible for the deaths of those 100 volunteers who signed up and got placebo'd and died.
Some people are comparing this to the trolley scenario, but the key distinction is that those 100 dead people are volunteers, who presumably understand the risks. Now the burden falls to you: why are you blocking volunteers from risking their life to save 399,900 people? Are those deaths now "unnatural"? Or are they now directly on the hands of the people who made this miscalculation?
A lot of people in this thread have made a similar argument so I'll try to respond to that here. I'd argue that 400k deaths (in the US alone, not globally) represent a massive policy failure. Or even more explicitly, it was a policy *preference* of those in power (the "let's achieve natural herd immunity" philosophy). I don't think you can argue that a challenge trial would have prevented those deaths unless you can argue that government officials would've been more prone to provide restaurant relief, enforcd mask mandates, increase unemployment insurance, etc. with an earlier vaccine (because they clearly haven't been willing to do so up until now).
More importantly though, the ethical angle looks a lot different in spring/early summer 2020, when such a decision would've been fraught with more unknowns. There simply wasn't the data on the nature of COVID (how to treat it, long term effects, who's more prone to infection, etc.). And again, we know that even people who recover are prone to heart problems and other long term effects. The nature of a challenge trial is philosophically similar to "right to try" advocates for drug trials (which is a larger discussion, but it can be argued that this could lead pharma to turn into the herbal supplements industry). You can tell people that these are the risks, but the picture looks a lot different when it's 3-5 years later (or more) and you're still not fully recovered.
I'd also argue that the "run a challenge trial" proponents need to think carefully about study design. How many people are you running this with? How are you exposing them to the virus? Masked or unmasked? What's the viral load? What's the strain? How long are you exposing them? How are you preventing them from being exposed outside of the confines of the study? How would we measure vaccine safety on a smaller population? A proper challenge trial study would need to answer all these questions and argue that getting to an answer would happen faster than the incredibly fast development we've seen with the Pfizer and Moderna vaccines (which met their targets of a Phase 3 readout in mid-fall 2020).
I think everyone agrees that the actual harm of Trump administration policies is (1) really bad and (2) a total failure for which we could have done better.
But I think the argument for challenge trials isn't that "we need them because we suck at mitigation", it's that the non-challenge trial needs a certain amount of naturally occurring harm to happen to the placebo group in order to demonstrate efficacy. So the harm of the non-challenge trial is either going to happen quickly (because we have crap mitigation) or slowly (because we have good mitigation) but it's going to happen, and the population itself may be suffering harm outside the study in parallel.
The only way around this I see that doesn't involve challenges is to relax the requirement for a double-blind demonstration of efficacy vs placebo by observing that the placebo group becomes ill.
I think I'm not clear on whether your argument is "you can't really have informed consent" or "this is deontologically immoral" (e.g. it's never okay to go infect people on purpose, regardless of how much harm you think you're fixing) or "there are too many unknowns to safely make this harmful action on utilitarian grounds?"
Yes, the studies clearly went faster because of the exploding infection rates. I guess the counterfactual is, in a world where we were mitigating properly, how quickly would it have taken to get a readout? The estimates were 3-4 months longer. So with much lower infection rates, we still would've had approval in early spring 2021, with production already having been ramped up to the point where most of the population could receive their first dose right away. That would be a much better scenario than our current one!
To respond to your question, my feeling is "there are too many unknowns to safely make this harmful action on utilitarian grounds" as you say. I think challenge trials have a place, but not with COVID. Proponents need to argue more clearly how it would've worked because I don't see how lives would've been saved (either in our current world or a world where the mitigation efforts had succeeded more broadly).
Thanks - that's clarifying. Would you say "challenge trials have a place but not with COVID _ever_" or "challenge trials have a place but not with COVID if it were managed non-stupidly?"
In other words (and this is a very, very hypothetical question because if I'm giving you a magic do-over card for how you think things should have gone, why can't you snap your fingers and go "competent mitigation") if it were a sadly fixed and foregone conclusion that we were going to have rampant community spread and hundreds of thousands of casualties, would that (own-goal pandemic) situation be fare game for challenge trials? Or still too much unknown?
I guess I go back to two key points here: (a) the long-term effects of COVID are potentially dangerous enough that we should be very, very careful about exposing people, and (b) I have yet to see a challenge trial design that gets us to fewer deaths/infections than our current state.
Essentially, you're saying "we would have willingly exposed X number of people to COVID in a challenge trial to save Y lives" knowing that 400k US deaths would have happened by now. In such a hypothetical universe, yes you can make that argument. But the particulars matter here. It's an amazing scientific feat that we got 95% efficacious vaccines into people 10 months after COVID was sequenced. Would it have really happened faster a different way?
Were the estimates really only 3 to 4 months longer to gather the data? If everywhere was Australia or New Zealand would it not take years? I think the key point of this entire discussion is that it took only a matter of weeks to develop a vaccine candidate (how smart are those people!) and the rest was just testing. We should be considering every possible approach to speed up testing and having debates about the ethics and science - which this message board is doing. Almost by definition the ethics will never be 'settled'.
Of course with a more definite and shorter timeline there would have been more political will for stronger NPIs.
But even with the same level of NPIs, getting a vaccine a day sooner would've saved thousands of lives. That should have huge moral weight. The ethical issues involved in letting *willing volunteers* get infected with covid are much smaller; maybe dozens of lives rather than thousands.
It's true there were (and are) a lot of unknowns about the effects of covid, but those same unknowns also weigh on the other side of the scale; by delaying a vaccine, you subject thousands of people to those unknown effects as the pandemic plays out naturally.
The questions about the design of the study are real, but it's hard to believe they couldn't have been figured out in a couple days. But we never got past the ethics issues to see.
Getting a *vaccine* a day sooner would have saved thousands of people. But getting *knowledge* of vaccine effectiveness a day sooner wouldn't have saved thousands of people, because vaccine distribution has been hampered more by manufacturing and logistics than by knowledge of effectiveness.
We didn't wait for knowledge of effectiveness to start manufacturing candidates, and we didn't have to wait for knowledge of effectiveness to start hiring injection workers and designing signup websites. It seems likely that instead of 14 million doses manufactured and 5 million given by the end of December, we might have had 2 million manufactured and 1 million given by the end of November, and 15 million manufactured and 8 million given by the end of December. But if we actually continue to ramp up injection capacity, then likely in a few weeks we will reach the same manufacturing bottleneck that we just would have reached sooner.
This is a good point.
I'm not sure these issues are as disconnected as you'd hope. For instance, the vaccine rollout was poor even though people had months to prepare. Prep time isn't as good as the real thing. I imagine it's psychologically (and probably legally?) easier to work on mass producing something which is already approved vs. speculative. But I'm not sure how big this effect is. I would love to read a breakdown of the manufacturing timelines and whether they were as fast as possible.
There is also some benefit to actually getting the doses out sooner (reduces more future infections).
This makes total sense to me, but then I've never seen the trolley problem as a difficult problem. You throw the switch, save net lives, sleep like a baby that night. There seems to be this human idea that sins we cause through inaction to be less bad than sins we cause through inaction, which I will never understand.
FWIW, what ethicists traditionally consider to be the "trolley problem" is not "what do you do when you see a switch that can kill one to save five?", but rather "why is it perfectly good to push a switch to kill one to save five, when it would be awful to shove one fat person physically onto the tracks to stop the trolley and save five?" There are many versions of the "kill one to save five" scenario that are discussed in the literature, and most people share the intuitions that in some of them you absolutely should do it, while in others you shouldn't.
(My take is that the examples are badly designed, and people are reacting to the fact that in the real world, the scenarios that people react badly to are ones in which you would be very far from certain that killing one would actually succeed in saving five - of course you shouldn't kill one for a small chance of saving five!)
That's a good point, a key difference between the trolley problem and real life is in the hypothetical scenario you have perfect information and there is no random chance involved.
Under what conditions are challenge trials going to be allowed? In policy, a clear bright line like "under no circumstances should you make volunteers intentionally sick" is valuable. This is effectively Matt's argument with vaccinating by age and the same argument applies with study design.
But the counterfactual world isn't a world where those people DON'T get the virus. In fact, it very deliberately is a world where they DO get the virus, since that's the only way to get the vaccine approved!
So... the argument is we should intentionally infect people in a controlled environment since people are getting it anyway?
I mean, yes, the vaccine readouts came much faster because the infection levels were so high. But that's not really a good argument to do challenge trials! If anything, it's an argument against, since you're not going to be able to ensure that people in such a challenge trial wouldn't be exposed outside of the trial guidelines.
Also, as others have pointed out, you don't get proper safety readouts without a placebo arm. And as we've seen, there are expected side effects. You can't analyze these without a control group for comparison.
Almost. I think the argument is that we should intentionally infect a _small_ number of people because a _large_ number of people are going to get it anyway. If non-challenge trial takes longer, the ethical cost of the non-challenge trial is what goes on in the rest of the community during the long trial if it turns out that the vaccine works (which has a > 0 expected value).
Not exactly... the cost is how long it takes to get people vaccinated. Which as we've seen, depends on manufacturing and distribution (not trivial considerations). Would any company have been able to produce more vaccines right now if they had had their readout 2-3 months earlier? Doesn't seem like that was the rate determining step, seeing as how they all ramped up production well before approval.
Right - you can argue against a consequentialist case for challenge trials on pragmatic grounds, but that argument can actually be settled in a way the philosophical one cannot.
Yes, they almost certainly would have been able to ramp up manufacturing earlier if they could expect approval earlier.
Evidence? The companies ramped up production at risk pretty early on in this case.
My understanding (from when I volunteered for the challenge trial) is that participants are kept in isolation, ensuring that they won’t expose others and also giving quick access to medical attention.
Not much for utilitarianism, huh?
Derek Lowe summarizes his argument as:
> I think that such trials are (1) still ethically questionable, (2) fraught with legal complications, (3) are for those reasons and others not likely to run meaningfully faster than regular Phase II trials, and (4) will produce data that may not be able to be extrapolated to the real-world situation.
Putting aside (1) as a question that will depend on your personal ethics, (2) and (3) are both about COVID-19. Matt is proposing to invest in solving those problems in advance of the
*next viral pandemic*. That proposal is entirely consistent with Derek Lowe's points.
(4) is a genuine factual disagreement. Derek Lowe presented an argument, and Matt presented a counter-argument. If you (@J. Willard Gibbs or others) don't think that counter-argument is convincing, why not?
A fair question and thanks for laying it out this way. My feelings on (1) seem to be counter to a lot of the folks here (there's a larger conversation on "right to try" drugs that needs to be had). I think much of the argument in this thread surrounds (2)/(3) but (4) to me is really the crux of it. So I'll state here as I have in other responses -- challenge trials have a place, but they are very, very situation-specific and COVID doesn't fall into that paradigm.
I guess surrounding (2)/(3) -- could we have actually developed something faster than we did using a challenge trial? Because we got these vaccines incredibly fast, to the point where manufacturing is going to be the holdup in getting people dosed and not the trials themselves.
But (4) is the big problem you have to solve. How, exactly, do you design a challenge trial that gets enough doses of a safe, efficacious vaccine into people more quickly than a traditional trial? Because every disease is a little bit different. You can use learnings from a previous vaccine (and with mRNA, we should!) but there's no guarantee that it will be safe or efficacious until you test it.
Traditional phase 3 (non-challenge) trials run the same set of risks, you just don't know what the confounding variables are. What was the weather like during the "real world" trial? What were local infection rates at the time? What was the market price of grapefruit? Do any of these matter? Are either of them divergent from the "real world" conditions that the vaccine will be deployed to the general public in?
The point of a traditional phase 3 non-challenge trial is that by including 30,000 people from a bunch of different sites in different nations in different hemispheres, you can create a study population that approximates the world average in a very large number of characteristics. Importantly, it approximates the world average in terms of the type and duration of exposure - which challenge trials very much would *not* (unless we've already done other trials to determine what sorts of exposure really have been the relevant ones in the world, which is something we absolutely should do, but would likely take longer than a year).
Exposure for the actual phase 3 trials happened under conditions where masks were mandated, many businesses closed, etc etc. It's not the same problem to the same extent, for sure. But I just don't see why the evidential bar has to be so high, for the FDA to just stop banning a medication and let people take it if they want. I think the "right to try" issues are really relevant to the question of the burden of proof here.
I apologize. No pissing match intended. I will try to back off of any rancorous tone I had.
I will not attempt to devise a specific study design. I am not a field expert. But I was under the impression that Lowe opposed Challenge Trials not due to implementation problems, but because he ethically feared exposing participants to a virus with unknown consequences. I’m sure there is an equally interesting and complicated debate to be had about how to properly implement a Challenge Trial but I am not doing so here.
I used the term “In the Wild” to somewhat glibly refer to the Non-Challenge Trial process. My understanding is as such:
- vaccinate some volunteers, call them Group V
- placebo some other volunteers, call them Group P
- let them all run around in the wild for a while
- once some portion of Group P have contracted the virus, this indicates there’s probably been sufficient exposure to Group V
- then check Group V to see if they fared better. If they did? You got yourself a vaccine.
In this sense, in a Non-Challenge trial, placebo participants are still getting exposed to the virus.
The only difference is it is passive exposure rather than active exposure. In a Challenge Trial you _actively_ expose them in a lab, in a Non Challenge Trial you _passively_ expose them on the streets of cities and towns across the world.
This is the first important point: you’re not “saving” any placebo participants from contracting the virus. For any statistically meaningful result (vaccine good? or bad?) some number of people in Group P is getting the virus.
The second important detail is that in the Non Challenge Trial, you require several non-participants, call it Group N, to also get the virus. These are not volunteers. These are ordinary people on the street. It is only through their exposure and transmission that you ever get any results in Group P or Group V!
For any Non-Challenge vaccine to come about, you must involve this non-volunteer Group N! Worse, Group N has to be _massive_ for there to be any statistically meaningful contact with Groups P and V.
When put in these terms, Non-Challenge Trials sound downright vicious! Not only must they involve non-volunteers, they involve _tons_ of them!
Thus, to many of people in this comment thread, they think who but a monster could oppose a Challenge Trial?! (again, assuming you could figure out implementation)
But indeed I don’t think you’re a monster. Nor Lowe. Here is what I think the core of this debate centers around and why many people seem to be talking past each other (and why I think this is such an interesting subject). We have different ethical frameworks and we think differently about active vs passive harm.
It sounds like to some people, the thought of actively harming Group P is worse than the thought of passively allowing harm to come to Group N. At the end of the day, I think this entire debate is that simple.
Personally, I would rather actively harm Group P (assuming they were volunteers) than passively allow harm to come to (a much larger) Group N of non-volunteers. That’s why I like the idea of Challenge Trials.
Okay, I'll bite:
Let's say (roughly) there were 30,000 people involved in a traditional trial. So 15k in the placebo group, 15k in the vaccine group (this is roughly the breakdown for Moderna/Pfizer/J&J/AZ). To achieve statistical power, the study designers determined that they would check endpoints when 30, then 60, then 90, then 120 people were infected across BOTH groups (again, roughly). Then they would compare the figures of people in who would be infected and measure the efficacy based on the ratio of people infected. So for example, if 80 people were infected in placebo group and 8 in the vaccine group, that's 90% efficacy (again, these were roughly the numbers you saw in Pfizer/Moderna).
Now if you ask me whether you're putting those 80 people at risk -- yes, you are! But now let's say you run a challenge trial and determine you need 10,000 people to achieve statistical power (I'm completely pulling that figure out of thin air... I don't know the actual number you'd need). So you expose 10,000 people to the disease, 1,000 get infected -- 90% efficacy. You've proven the same thing but exposed many more people to the disease. Obviously the numbers are arbitrary here... but are we really going to be able to get a safety readout amongst a cross-section of society with 100 people? Probably not. What about 1,000 people? Unlikely, but let's say for the sake of argument that you can -- that's still 100 people infected and you're greater than your placebo group in the non-challenge trial!
I just think that thinking about this as a trolley problem is getting a little too cute. You need a real world trial design that addresses these issues. It's not enough to say you're harming people passively versus actively -- I just don't see how a challenge trial reduces the harm at all.
I don’t see how you ever can expose fewer people to the virus in non Challenge scenario.
Suppose, as you say, you need ~120 placebo people to have contracted the virus before you’ve got yourself a meaningful sample. At this point you assume ~120 vaccine people also were exposed and so you’re saying you need an exposure group of ~240 people to make any meaningful determination.
You have two ways of exposing these people:
- expose each in an isolated lab (Challenge Trial)
- tell each to walk around NY until you’ve reached that 120 number
Practically by definition, the latter one is going to require that more than 240 people were exposed! By the time you’ve hit your 120 mark, thousands of New Yorkers would needed to have it in order for it to jump to those 240 people.
The numbers factor out. Call 120 N. For any N (short of some weird edge case where N is the size of the planet’s population) you’ll always limit exposure by doing the exposures in a lab rather than doing the exposures in the wild and waiting and hoping.
Except that you're only considering efficacy, not safety. You couldn't get enough of a safety readout on a vaccine with only 240 people getting it.
Then pick a higher N.
If you pick an N of 10,000 you will still require less global exposure by exposing each individually than if you sit around and wait for them to be naturally exposed by the rest of the world. Again- because for them to be naturally exposed millions of non-participants will need to get it.
You raise an interesting point about safety vs efficacy that I hadn’t considered. I suppose the goal there is to vaccinate enough people that you understand the dangerous side effects are under some low bar.
I don’t think this presents any categorically new problem in a Challenge vs a Non Challenge:
you can still choose to vaccinate as many as you want/need and you pick some lower subset of the group to expose.
The vaccinated but non-exposed people give you a safety readout. The vaccinated but exposed people give you an efficacy readout. You cross both with a placebo group so you have a baseline for both.
But does any of that change the fact that you’re still at the end of the day being more _efficient_ about exposure (thereby exposing fewer total people)?
Well now you have two arms to your trial, a safety arm and an efficacy arm. And you've complicated matters immensely. Is this going to provide a cleaner statistical readout than either a straight challenge trial or a traditional trial? And is it going to get you to your answer faster?
Again, Pfizer and Moderna both managed to produce safe vaccines that were 95% efficacious in 10 months, which beat the previous record for a vaccine by over three years. They were so fast that they couldn't even ramp up manufacturing to have enough doses ready by the time of approval (pharma thinks about this in great detail, and Pfizer's CEO Albert Bourla was very clear in the summer that they had already started manufacturing at risk). The onus is really, really high for designing a challenge trial that gets you where you need to be faster.
Right... and now more people have it.