Again it all seems obvious until you design the experiment. “Just expose people to the virus” does a lot of work for you but it by no means an experimental design. Please interview a scientist on how they would choose to do this experiment and your argument will be stronger. Do you put the test subject in a room with sick people? Inject it? Gas people with it? How large is the exposure? When you do it the standard way, people are exposed to “normal” levels of the virus not some standard that must first be scientifically established. Not all viral infections are the same, quantities and methods matter.
Agree 100%. I'm a biostatistician and the design of the Pfizer/Moderna trials are dead simple, based on decades of clinical experience and statistical research. An undergraduate statistics student could the study design.
For challenge trials, the design is way more complicated. Even determining what results from the trial would mean that it works is extremely hard, because that depends on how contagious the disease is, the exposure setup, and the effectiveness of the vaccine.
The FDA said they would approve any vaccine that is safe and at least 50% effective. I would love for a proponent of challenge trials to describe an experimental design that would assess the effectiveness of a vaccine with a true effectiveness of 50%. It's not a trivial problem!
This is not the attitude that got us to the moon. The specialists should have found a solution to this not-trivial problem. But clearly they felt no responsibility to do so.
Anyone can see that tests were not expedited in the best possible way. It's up to the specialists to figure out what would be a better way. Anyone can identify the problem, only a specialist can pinpoint the ideal solution. Why is that suprising?
Sure, but that takes time. The Pfizer and Moderna trials each lasted about two months after vaccination. Best case scenario, a challenge trial saves a few weeks? A month? Maybe an additional few weeks saved during the enrollment, which would also be smaller/faster.
Obviously, even a month or two faster would save a lot of lives. But any additional time spend researching study design would very quickly eat into those savings. The Pfizer/Moderna trials could be designed in a few hours, so if it takes a month to research and design the challenge trial, it's basically a wash.
I think the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics.
"the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics."
This seems like the place we should all end up converging. Hopefully the people who start thinking about this stuff now will end up coming to some sort of consensus about the ways in which challenge trials will and won't be a good thing in a future pandemic, and can come up with streamlined procedures for expediting the ones that should happen, and clarifying which ones shouldn't happen.
In the mean time, I think there should be a lot of research devoted to ecological challenge trials of respiratory viruses generally. It's only this past year that I finally realized how little we know about the relative role of surfaces, droplets, and aerosols in the transmission of various "cold" viruses and influenza, and about the relative effectiveness of interventions targeted at children or adults, including school closures, social distancing, and mask use.
This is an optimistic vision, but I'm afraid that the consensus you hope for will be forged by hidebound cover-my-ass/"maintain the public trust" FDA types like Eric Topol, Paul Offit, and so on.
Im interviewing to PhD programs right now do you think this is a good gateway to your field or is masters more the standard? Ive been in biomedical research for a while now and I’m wondering if my experience is sufficient already if I decided to go that route. Id like to do basic research regardless just worried about the long game.
If you want to do biomedical research, I'd say a PhD is standard. Quite honestly, a large portion of people who get masters in biostatistics in my department end up in data science jobs in tech.
This seems like a good sort of way to do it, but it seems far from obvious to me that you'd be able to both end up with fewer people infected and a shorter trial time. I guess each round of this trial would take at least a week, to find an infected and infectious person, use them to expose several volunteers, then keep the volunteers isolated until you can be sure they didn't get infected for that interaction. If you start with a large enough number of participants to be sure you'll get significance early, you'll probably be using a larger number of people than the standard Phase 3 trial used (since you're looking at just one afternoon of exposure, rather than the number of exposures that members of the public are averaging); and if you start with smaller populations until you get a sense of the exposure time needed, you'll end up needing to do many rounds (and the standard Phase 3 trials only ended up taking about ten weeks anyway).
These are surmountable study design and methodology questions that come into play once you get past threshold ethical objections to challenge studies at all. At most they come into the ethical discussion only insofar as they might undermine the premise that in a particular case a challenge study is more ethical because it would be faster and thus save lives.
I disagree in that I actually think the experimental design actually drives the ethics. For instance if I have to get a sick patient to cough on my test subject then Im potentially causing another human being to be responsible for the sickness of another who maybe is less qualified to understand the ethics or isn't in a good position to think about the consequences due to illness. I wouldn’t be surprised if there was an ethical way around these concerns but they need to be clearly stated. No amount of hand waving will make this core problem with Matts argument just go away.
That seems like just a routine question of informed consent, which comes up in every study involving human subjects. IRBs are very experienced at handling those. It might be harder to achieve informed consent for some challenge studies but that's not a good reason to categorically take challenge studies off the table.
If I might rephrase: challenge trials are a great way to test the efficacy of a vaccine against deliberate infection, but what we want is a vaccine that works in the real world. And we can't know if the former necessarily implies that latter, particularly with a novel virus.
With all do respect, I am an active, professional, PhD-having research scientist and I don't agree with that. The only way to establish that efficacy from deliberate infection maps directly on to real-world efficacy is to compare the two in rigorous studies, but in the absence of challenge trials, that study is not possible. I think Matt is out of his depths here because running challenge trials *now* would violate fundamental principles of the scientific method; at some point in the future they may become useful, but barring some huge cache of literature comparing the different methods, we're stuck with what we have for now.
Granted, my expertise is (bio)chemistry, so I am not an expert on infectious disease, but there are nuances to the method, frequency and level of exposure for many known viruses that are not strictly multiplicative. I see no reason that would not be true of the novel coronavirus. You would need to design a deliberate infection model that covers a sufficient combination of those (and other) relevant variables to be able to assume that it is sufficiently simulates real-world infection because every downstream assumption would inherent the flaws in the deliberate infection method. You would also need a sufficiently large group of humans to cover the natural variance in human immune responses, co-morbidities, etc.
It is entirely possible that I am entirely wrong, since the devil is usually in the details and I am not an expert. But at first blush, this is simply not how science works.
The way trials are currently designed, the statement that "there was an X% decrease in rates of infection with Y certainty compared to a placebo" is accurate. If you deliberately infect people as part of the trial, the statement narrows to "there was an X% decrease in rates of infection in people deliberately exposed to the virus with Y certainty compared to a placebo". Moreover, both groups would need to be held in controlled conditions for the duration of the study, or you would have to narrow the statement further to include the caveat that you did not control for random exposure.
"The relationship between vaccine efficacy and infection efficacy on success rate is *strictly multiplicative*. A vaccine that reduces prevents 63% of infections against a 70% infection rate is just as efficacious as one that prevents 81% of infections against a 90% infection rate: 90% effective."
I think this is technically correct, but not actually informative. We *define* vaccine efficacy as the ratio of number of infected with the vaccine to number of infected without the vaccine, in two populations with the same other characteristics (including distribution of exposure). But the efficacy is an efficacy *for that population*, and not in general.
There's no particularly good reason to think that a vaccine that has an observed 90% effectiveness in a population that is constantly kissing each other would have the same observed effectiveness in a population that is sharing needles, or a population that is grocery shopping with masks. If the vaccine works by creating a certain level of immune response, it could be that this immune response is 100% effective against viral doses that are below a certain size, but not against doses that are above that size (if they manage to overwhelm the response and infect many cells before the body gets it under control), so that it is much more effective for the grocery store population than for the IV drug population. It could be that the immune response is concentrated in the lymph fluid rather than the bloodstream. It could be that the immune response prevents a mechanism the virus uses to infect blood cells but not lung cells, so that it's more effective for the IV drug users than for the grocery shoppers.
Another way to put this is that in addition to whatever ethical problems they may or may not have, challenge trials have epistemological problems. They give you knowledge about how the vaccine does or doesn't protect against certain styles of lab-based intentional exposure, which doesn't give you much clear sense of whether the vaccine does or doesn't protect against the real-world kinds of exposure people actually get.
However, a friend pointed out to me that once we are considering challenge trials, there's room for them in a *lot* more than just the vaccine side. Right now we are doing all sorts of NPIs (non-pharmaceutical interventions), which are the medical euphemism for things like closing bars, telling people to wear masks and stay outside, and so on, but we have basically zero data as to whether or not these interventions are effective, and if so, how effective.
Once it was clear that this virus was not very harmful to 20-somethings, it seems that it would have been a good idea to do some challenge trials to measure these things. Get a few hundred volunteers to hang out in a bar for science, and do testing first to ensure that most of them tested negative, and a few tested positive, but don't tell them who is who in advance, and see who tested positive afterwards. Then do the same thing, while telling everyone to stay six feet apart, and requiring the bartenders to wear masks. Do the same thing with concerts and grocery stores and meat-packing factories and playgrounds and such.
If we had started this stuff in April, then maybe by September we would have had a better sense of how likely various types of exposure are to spread the virus, and how much different various interventions make to both individual and social risk. It might also have told us which social protocols are particularly effective ones at spreading, which we could then use in vaccine trials a month or two later, rather than relying on unrealistic injection scenarios.
Of course, one thing that understandably raises huge ethical concerns about this is that this is basically the study design of the Tuskegee syphilis experiment - get a bunch of people, test them, don't tell them the results, and then see how the disease progresses and spreads to others when untreated. But if you are working with a known low-risk population, and give everyone information a week after the experiment, and good treatment, and informed consent before the trial, you can mitigate a lot of that.
This seems like a minor technical issue (mainly because if the vaccine works, it will work against any method of exposure, and vice versa). I guess just try to emulate some known superspreader event for each participant?
I think one natural experiment one could do would be to compare the rates of death of healthcare workers to that of the average population. My hypothesis is that they die at higher relative rates than their age cohorts. If so this could demonstrate the way in which heightened viral exposure increases the mortality rate. I suggest this because it would show that a higher environmental dose results in more deaths. The point is we would need to settle on what is a typical dose this all takes time as well. Totally get what you are saying but this is probably how you would have to get at the question of what dosage emulates a super spreader event. I think it’s possible but may be a bit trickier than most imagine.
Further complicating lab infection: there is precedence for the sensitizing effects of a vaccine creating potentially worse outcome in a lab exposure than in wild exposure. The truly informed consent becomes harder because a subject could reasonably expect that the lab infection would mirror a real world infection when it could be much worse not only because of the issues you raise (strain, load, route) but because of the vaccine itself.
This is a great point. Even with COVID19 after all these months, there is considerable uncertainty about the "viral dose" someone receives out in the world and how sick they are likely to get. It seems to be mediated by masks, where masks reduce the dose and thus result in a mild infection. But, there is also quite a bit of variation in how strongly someone's immune system will react to infection. For a truly novel pathogen where the route of infection is not thoroughly understood, it would be difficult to design the challenge trial.
I’m all for challenge studies to get a quick read on efficacy but I don’t see that doing anything about Phase 3 studies. Vaccine Phase 3 studies are not the size they are simply because of efficacy. It’s because you need large trials to identify safety signals that are too rare to pop up in a small Phase 1 trial. Covid is bad but most people haven’t had it and the vast majority of people survive. The vaccine is going to go into *everyone on the planet* in an ideal world. The risk/benefit ratio is hugely critical for a scenario like this, and it’s very challenging to track adverse events in a non-controlled environment. To avoid Ph3 studies you would need a new way to accurately track safety in the general public.
You can separate out the need to do Phase 3s to test efficacy and the need to test safety and just run big safety trials. Here's a proposal along these lines from early in the pandemic related to challenge trials. -- https://www.medrxiv.org/content/10.1101/2020.05.18.20106187v2
This argument is much more logical than the "challenge studies are inherently unethical" response, which is another flavor of the same general story of medical ethicists being totally divorced from the actual study of regular ethics.
Right. We don't talk about this much - maybe because there's the crazier "vaccines cause autism" meme out there - but if you give an intervention to a young population planet-wide, the side effects have to be really close to zero to make it not a problem.
I think this is also a risk we eat no matter what to some level - the youth of today are the longevity test group.
At first I recoiled, but you convinced me on the merits. We have an all volunteer army risking their lives to protect the rest of us and that doesn't seem unethical. At sporting events we could cheer volunteers who risked their lives in challenge trials and offer our gratitude and collective sorrow at funerals for those given placebos. Seems odd, but better than the alternative of greater death.
We also allow people to undergo abdominal surgery and donate one of their kidneys or part of liver. This is a medical procedure with no benefit to the donator that carries some risk of bad outcomes including death. It seems like a very good analogy to me.
>>We let people volunteer to get shot at for the common good, and as bad as COVID may be, for most people it's a lot less risky than getting shot at.
Yes, but... That shouldn't be the crucial point here. Because it's even more crucial that we be willing to use these methods with deadlier diseases. With something like the MEV-1 virus from Contagion, slowing down trials would mean killing millions.
My question: do challenge trials exist in other countries that emphasize the collective over the individual? I am sure an apples-to-apples comparison is challenging considering situations like the suspect trials of Russia’s Sputnik vaccine.
Well, it's a variant of the trolley problem where you get to yell to the person that you're considering switching the track to, "Hey, are you okay with this," and he yells, "Yes!"
Well, no, that person is going to be very very very very likely to be sick and not dead, but there is a very small risk that that person will be dead.
I think the utilitarians are very comfortable with that.
I have a pet theory that humans round all risks up to 100% or down to 0%. Maybe this creates a problem for informed consent? I'd still let the volunteers take the risk, backed by what medical support we can give them.
Saying that people round risks up or down seems appealing for a lot of examples, but it doesn't explain the weird way that people overestimate risks from airplanes and bike riding and underestimate risks from cars and infectious disease. I think familiarity is also said to play a major role in people's estimate of risk, as well as whether the risk is thought to be something that the individual controls or is controlled by someone else.
Totally spot on. I work in aviation more or less, and we get absolutely crazy when non-aviation people get cranky about the risk. The indignation usually goes something like "how can you say that and get in a car??!?!"
Two limitations of the challenge model that challenge trial proponents rarely if ever address.
1. What is needed is a vaccine that prevents infection from real-world exposures to the virus. A challenge trial exposes patients to the virus in a different way, everyone getting a set amount via the same type of exposure. So a challenge trial should be great for clinical proof of concept, measured efficacy in a challenge trial may differ from real-world effectiveness of the same vaccine. Expose people to not enough virus and you may overestimate real-world effectiveness; epose them to too much and you may understimate vaccine potential and even reject an effective vaccine.
2. Challenge trials would be smaller and faster, which means conserably less safety data.
Both Moderna and Pfizer vaccines essentially charged right into Phase 3 without the typically slower process of first showing some efficacy in Phase 2, helped by the government's guaranteed purchase. Either or both of these vaccines could have failed in Phase 3, as many drugs do. Challenge trials would offer the advantage of weeding out ineffective drugs quickly.
I have nothing to add here; I just want to know the most plausible way to make challenge trials happen. I think the best strategy might be persuading the government of Russia or China to do them.
The worry about this is that if Russia or China do something that is judged to be as morally bad as the Tuskegee syphilis experiment, or the Nazi medical trials, then they could set back the whole cause of getting challenge trials approved. You want to be able to show people that it is possible to do challenge trials in a way that appropriately takes the ethical concerns into account, not just show that challenge trials are possible.
I don't think anyone has proposed doing challenge trials on people other than consenting adults. Matt had the patience to write a lot of words but TL;DR: it's frankly bizarre that letting consenting adults put their health at risk for the good of humanity is considered ethically dubious.
The problem with doing it in China, for example, is making sure that all the volunteers actually volunteered. I don't just mean to impugn China here, but also to note that the international press does not presume good will on the behalf of the Chinese and might end up concluding (whether correctly or not) that people were forced to take part.
I lived in China for more than a decade. It may be a police state but it's not a forced-labor camp (okay, maybe parts of Xinjiang). If the international press wants to find out whether subjects in clinical trials are participating of their own free will, they should interview them and see what they say. To make that possible the government would need to be more transparent than it normally prefers, but it's not a heavy lift.
Yes, I agree... part of my point is that to some extent the Western press is biased against China, and that could lead to problems with the Western public perception of trials done in China.
Sigh. No. The main ethical reason for not doing challenge trials is that COVID-19 is a new disease whose long term effects are unknown. Back in March or April, there was essentially no effective treatment; now at least, hospitals have figured out how to reduce the fatality rate somewhat (assuming they're not overrun with patients). But this is not a disease you want to have!
Is the sighing necessary? This “why are you asking such a dumb question” tone has been the dominant one from challenge trial opponents in the discussion I’ve seen and it smacks of “I was taught this was not an option years ago and I’m not confident in my ability to explain why.”
Sorry... the sigh is because this argument has been refuted by others over and over and it's frustrating to have to keep at it.
To be clear, because I guess I haven't been here: I don't think challenge trials are *never* an option, just that (a) the uncertainty around the science with COVID suggests that they would've been a bad idea here, and (b) challenge trial proponents need to be a lot more clear in how the study design lends itself to faster vaccinations. The details matter, a lot.
I admire you for standing for the opposing position and taking the heat from Matt's fanbase but your position here is completely unconvincing to me.
The status quo got alot of people killed or very very sick and has made those of us with high risk loved ones live in constant fear for almost a year now.
IMO public health officials need to do a better job of explaining why allowing all that death and suffering was better than deliberately infecting volunteers, because I find Matt's argument very compelling. I'd be willing to volunteer if it meant my loved ones got a vaccine quickly and would not get sick especially if government gaurenteed good healthcare and death benefits in the event that I got sick or died.
The *status quo* didn't get a lot of people sick; deliberate choices by political leaders to achieve "natural herd immunity" did. Matt did a good job laying out the NPIs and I don't need to discuss the politics of that side of it.
The problem that challenge trial defenders need to answer, as I've stated below, is: what sort of trial design gets you to safe, efficacious doses given to enough people to achieve herd immunity the fastest? Pfizer and Moderna each produced a 95% efficacious vaccine in 10 months! That's three years faster than any previous vaccine. They did it so quickly that manufacturing hasn't been able to keep up with demand, even though they both ramped up manufacturing this summer. So where would the time savings come into play with a challenge trial?
The time savings are abundantly clear. Even if only 3% of the nation had been slowly administered in September/October/November that would have saved a huge number of lives both directly and a lowering of the R0 before the weather got cold.
Every single political argument we are having now about speed/priority is peanuts compared to the huge amount of lives that would have been saved if the system was designed for time-sensitive pandemics.
Sorry, the refutation isn't very convincing. Millions of people have been infected with the coronavirus. Infecting a few thousand on purpose to validate a vaccine seems like a no brainer. And it doesn't need to be 50-50 vaccine and placebo.
I don't buy Lowe's argument. The core of his argument, as you say, was that the virus may have been really bad. If anything, this only boosts the Yglesias angle.
The question boils down to: is our test population the full globe? Or is our test population those volunteers who sign up?
You seem to be saying you'd rather the test population be the full globe because... the virus might be really bad?
I think there's some ethical view which says: we're not responsible for the deaths of 400,000 because those are "natural" whereas we would be directly responsible for the deaths of those 100 volunteers who signed up and got placebo'd and died.
Some people are comparing this to the trolley scenario, but the key distinction is that those 100 dead people are volunteers, who presumably understand the risks. Now the burden falls to you: why are you blocking volunteers from risking their life to save 399,900 people? Are those deaths now "unnatural"? Or are they now directly on the hands of the people who made this miscalculation?
A lot of people in this thread have made a similar argument so I'll try to respond to that here. I'd argue that 400k deaths (in the US alone, not globally) represent a massive policy failure. Or even more explicitly, it was a policy *preference* of those in power (the "let's achieve natural herd immunity" philosophy). I don't think you can argue that a challenge trial would have prevented those deaths unless you can argue that government officials would've been more prone to provide restaurant relief, enforcd mask mandates, increase unemployment insurance, etc. with an earlier vaccine (because they clearly haven't been willing to do so up until now).
More importantly though, the ethical angle looks a lot different in spring/early summer 2020, when such a decision would've been fraught with more unknowns. There simply wasn't the data on the nature of COVID (how to treat it, long term effects, who's more prone to infection, etc.). And again, we know that even people who recover are prone to heart problems and other long term effects. The nature of a challenge trial is philosophically similar to "right to try" advocates for drug trials (which is a larger discussion, but it can be argued that this could lead pharma to turn into the herbal supplements industry). You can tell people that these are the risks, but the picture looks a lot different when it's 3-5 years later (or more) and you're still not fully recovered.
I'd also argue that the "run a challenge trial" proponents need to think carefully about study design. How many people are you running this with? How are you exposing them to the virus? Masked or unmasked? What's the viral load? What's the strain? How long are you exposing them? How are you preventing them from being exposed outside of the confines of the study? How would we measure vaccine safety on a smaller population? A proper challenge trial study would need to answer all these questions and argue that getting to an answer would happen faster than the incredibly fast development we've seen with the Pfizer and Moderna vaccines (which met their targets of a Phase 3 readout in mid-fall 2020).
I think everyone agrees that the actual harm of Trump administration policies is (1) really bad and (2) a total failure for which we could have done better.
But I think the argument for challenge trials isn't that "we need them because we suck at mitigation", it's that the non-challenge trial needs a certain amount of naturally occurring harm to happen to the placebo group in order to demonstrate efficacy. So the harm of the non-challenge trial is either going to happen quickly (because we have crap mitigation) or slowly (because we have good mitigation) but it's going to happen, and the population itself may be suffering harm outside the study in parallel.
The only way around this I see that doesn't involve challenges is to relax the requirement for a double-blind demonstration of efficacy vs placebo by observing that the placebo group becomes ill.
I think I'm not clear on whether your argument is "you can't really have informed consent" or "this is deontologically immoral" (e.g. it's never okay to go infect people on purpose, regardless of how much harm you think you're fixing) or "there are too many unknowns to safely make this harmful action on utilitarian grounds?"
Yes, the studies clearly went faster because of the exploding infection rates. I guess the counterfactual is, in a world where we were mitigating properly, how quickly would it have taken to get a readout? The estimates were 3-4 months longer. So with much lower infection rates, we still would've had approval in early spring 2021, with production already having been ramped up to the point where most of the population could receive their first dose right away. That would be a much better scenario than our current one!
To respond to your question, my feeling is "there are too many unknowns to safely make this harmful action on utilitarian grounds" as you say. I think challenge trials have a place, but not with COVID. Proponents need to argue more clearly how it would've worked because I don't see how lives would've been saved (either in our current world or a world where the mitigation efforts had succeeded more broadly).
Thanks - that's clarifying. Would you say "challenge trials have a place but not with COVID _ever_" or "challenge trials have a place but not with COVID if it were managed non-stupidly?"
In other words (and this is a very, very hypothetical question because if I'm giving you a magic do-over card for how you think things should have gone, why can't you snap your fingers and go "competent mitigation") if it were a sadly fixed and foregone conclusion that we were going to have rampant community spread and hundreds of thousands of casualties, would that (own-goal pandemic) situation be fare game for challenge trials? Or still too much unknown?
I guess I go back to two key points here: (a) the long-term effects of COVID are potentially dangerous enough that we should be very, very careful about exposing people, and (b) I have yet to see a challenge trial design that gets us to fewer deaths/infections than our current state.
Essentially, you're saying "we would have willingly exposed X number of people to COVID in a challenge trial to save Y lives" knowing that 400k US deaths would have happened by now. In such a hypothetical universe, yes you can make that argument. But the particulars matter here. It's an amazing scientific feat that we got 95% efficacious vaccines into people 10 months after COVID was sequenced. Would it have really happened faster a different way?
Were the estimates really only 3 to 4 months longer to gather the data? If everywhere was Australia or New Zealand would it not take years? I think the key point of this entire discussion is that it took only a matter of weeks to develop a vaccine candidate (how smart are those people!) and the rest was just testing. We should be considering every possible approach to speed up testing and having debates about the ethics and science - which this message board is doing. Almost by definition the ethics will never be 'settled'.
Of course with a more definite and shorter timeline there would have been more political will for stronger NPIs.
But even with the same level of NPIs, getting a vaccine a day sooner would've saved thousands of lives. That should have huge moral weight. The ethical issues involved in letting *willing volunteers* get infected with covid are much smaller; maybe dozens of lives rather than thousands.
It's true there were (and are) a lot of unknowns about the effects of covid, but those same unknowns also weigh on the other side of the scale; by delaying a vaccine, you subject thousands of people to those unknown effects as the pandemic plays out naturally.
The questions about the design of the study are real, but it's hard to believe they couldn't have been figured out in a couple days. But we never got past the ethics issues to see.
Getting a *vaccine* a day sooner would have saved thousands of people. But getting *knowledge* of vaccine effectiveness a day sooner wouldn't have saved thousands of people, because vaccine distribution has been hampered more by manufacturing and logistics than by knowledge of effectiveness.
We didn't wait for knowledge of effectiveness to start manufacturing candidates, and we didn't have to wait for knowledge of effectiveness to start hiring injection workers and designing signup websites. It seems likely that instead of 14 million doses manufactured and 5 million given by the end of December, we might have had 2 million manufactured and 1 million given by the end of November, and 15 million manufactured and 8 million given by the end of December. But if we actually continue to ramp up injection capacity, then likely in a few weeks we will reach the same manufacturing bottleneck that we just would have reached sooner.
I'm not sure these issues are as disconnected as you'd hope. For instance, the vaccine rollout was poor even though people had months to prepare. Prep time isn't as good as the real thing. I imagine it's psychologically (and probably legally?) easier to work on mass producing something which is already approved vs. speculative. But I'm not sure how big this effect is. I would love to read a breakdown of the manufacturing timelines and whether they were as fast as possible.
There is also some benefit to actually getting the doses out sooner (reduces more future infections).
This makes total sense to me, but then I've never seen the trolley problem as a difficult problem. You throw the switch, save net lives, sleep like a baby that night. There seems to be this human idea that sins we cause through inaction to be less bad than sins we cause through inaction, which I will never understand.
FWIW, what ethicists traditionally consider to be the "trolley problem" is not "what do you do when you see a switch that can kill one to save five?", but rather "why is it perfectly good to push a switch to kill one to save five, when it would be awful to shove one fat person physically onto the tracks to stop the trolley and save five?" There are many versions of the "kill one to save five" scenario that are discussed in the literature, and most people share the intuitions that in some of them you absolutely should do it, while in others you shouldn't.
(My take is that the examples are badly designed, and people are reacting to the fact that in the real world, the scenarios that people react badly to are ones in which you would be very far from certain that killing one would actually succeed in saving five - of course you shouldn't kill one for a small chance of saving five!)
That's a good point, a key difference between the trolley problem and real life is in the hypothetical scenario you have perfect information and there is no random chance involved.
Under what conditions are challenge trials going to be allowed? In policy, a clear bright line like "under no circumstances should you make volunteers intentionally sick" is valuable. This is effectively Matt's argument with vaccinating by age and the same argument applies with study design.
But the counterfactual world isn't a world where those people DON'T get the virus. In fact, it very deliberately is a world where they DO get the virus, since that's the only way to get the vaccine approved!
So... the argument is we should intentionally infect people in a controlled environment since people are getting it anyway?
I mean, yes, the vaccine readouts came much faster because the infection levels were so high. But that's not really a good argument to do challenge trials! If anything, it's an argument against, since you're not going to be able to ensure that people in such a challenge trial wouldn't be exposed outside of the trial guidelines.
Also, as others have pointed out, you don't get proper safety readouts without a placebo arm. And as we've seen, there are expected side effects. You can't analyze these without a control group for comparison.
Almost. I think the argument is that we should intentionally infect a _small_ number of people because a _large_ number of people are going to get it anyway. If non-challenge trial takes longer, the ethical cost of the non-challenge trial is what goes on in the rest of the community during the long trial if it turns out that the vaccine works (which has a > 0 expected value).
Not exactly... the cost is how long it takes to get people vaccinated. Which as we've seen, depends on manufacturing and distribution (not trivial considerations). Would any company have been able to produce more vaccines right now if they had had their readout 2-3 months earlier? Doesn't seem like that was the rate determining step, seeing as how they all ramped up production well before approval.
Right - you can argue against a consequentialist case for challenge trials on pragmatic grounds, but that argument can actually be settled in a way the philosophical one cannot.
My understanding (from when I volunteered for the challenge trial) is that participants are kept in isolation, ensuring that they won’t expose others and also giving quick access to medical attention.
> I think that such trials are (1) still ethically questionable, (2) fraught with legal complications, (3) are for those reasons and others not likely to run meaningfully faster than regular Phase II trials, and (4) will produce data that may not be able to be extrapolated to the real-world situation.
Putting aside (1) as a question that will depend on your personal ethics, (2) and (3) are both about COVID-19. Matt is proposing to invest in solving those problems in advance of the
*next viral pandemic*. That proposal is entirely consistent with Derek Lowe's points.
(4) is a genuine factual disagreement. Derek Lowe presented an argument, and Matt presented a counter-argument. If you (@J. Willard Gibbs or others) don't think that counter-argument is convincing, why not?
A fair question and thanks for laying it out this way. My feelings on (1) seem to be counter to a lot of the folks here (there's a larger conversation on "right to try" drugs that needs to be had). I think much of the argument in this thread surrounds (2)/(3) but (4) to me is really the crux of it. So I'll state here as I have in other responses -- challenge trials have a place, but they are very, very situation-specific and COVID doesn't fall into that paradigm.
I guess surrounding (2)/(3) -- could we have actually developed something faster than we did using a challenge trial? Because we got these vaccines incredibly fast, to the point where manufacturing is going to be the holdup in getting people dosed and not the trials themselves.
But (4) is the big problem you have to solve. How, exactly, do you design a challenge trial that gets enough doses of a safe, efficacious vaccine into people more quickly than a traditional trial? Because every disease is a little bit different. You can use learnings from a previous vaccine (and with mRNA, we should!) but there's no guarantee that it will be safe or efficacious until you test it.
Traditional phase 3 (non-challenge) trials run the same set of risks, you just don't know what the confounding variables are. What was the weather like during the "real world" trial? What were local infection rates at the time? What was the market price of grapefruit? Do any of these matter? Are either of them divergent from the "real world" conditions that the vaccine will be deployed to the general public in?
The point of a traditional phase 3 non-challenge trial is that by including 30,000 people from a bunch of different sites in different nations in different hemispheres, you can create a study population that approximates the world average in a very large number of characteristics. Importantly, it approximates the world average in terms of the type and duration of exposure - which challenge trials very much would *not* (unless we've already done other trials to determine what sorts of exposure really have been the relevant ones in the world, which is something we absolutely should do, but would likely take longer than a year).
Exposure for the actual phase 3 trials happened under conditions where masks were mandated, many businesses closed, etc etc. It's not the same problem to the same extent, for sure. But I just don't see why the evidential bar has to be so high, for the FDA to just stop banning a medication and let people take it if they want. I think the "right to try" issues are really relevant to the question of the burden of proof here.
I apologize. No pissing match intended. I will try to back off of any rancorous tone I had.
I will not attempt to devise a specific study design. I am not a field expert. But I was under the impression that Lowe opposed Challenge Trials not due to implementation problems, but because he ethically feared exposing participants to a virus with unknown consequences. I’m sure there is an equally interesting and complicated debate to be had about how to properly implement a Challenge Trial but I am not doing so here.
I used the term “In the Wild” to somewhat glibly refer to the Non-Challenge Trial process. My understanding is as such:
- vaccinate some volunteers, call them Group V
- placebo some other volunteers, call them Group P
- let them all run around in the wild for a while
- once some portion of Group P have contracted the virus, this indicates there’s probably been sufficient exposure to Group V
- then check Group V to see if they fared better. If they did? You got yourself a vaccine.
In this sense, in a Non-Challenge trial, placebo participants are still getting exposed to the virus.
The only difference is it is passive exposure rather than active exposure. In a Challenge Trial you _actively_ expose them in a lab, in a Non Challenge Trial you _passively_ expose them on the streets of cities and towns across the world.
This is the first important point: you’re not “saving” any placebo participants from contracting the virus. For any statistically meaningful result (vaccine good? or bad?) some number of people in Group P is getting the virus.
The second important detail is that in the Non Challenge Trial, you require several non-participants, call it Group N, to also get the virus. These are not volunteers. These are ordinary people on the street. It is only through their exposure and transmission that you ever get any results in Group P or Group V!
For any Non-Challenge vaccine to come about, you must involve this non-volunteer Group N! Worse, Group N has to be _massive_ for there to be any statistically meaningful contact with Groups P and V.
When put in these terms, Non-Challenge Trials sound downright vicious! Not only must they involve non-volunteers, they involve _tons_ of them!
Thus, to many of people in this comment thread, they think who but a monster could oppose a Challenge Trial?! (again, assuming you could figure out implementation)
But indeed I don’t think you’re a monster. Nor Lowe. Here is what I think the core of this debate centers around and why many people seem to be talking past each other (and why I think this is such an interesting subject). We have different ethical frameworks and we think differently about active vs passive harm.
It sounds like to some people, the thought of actively harming Group P is worse than the thought of passively allowing harm to come to Group N. At the end of the day, I think this entire debate is that simple.
Personally, I would rather actively harm Group P (assuming they were volunteers) than passively allow harm to come to (a much larger) Group N of non-volunteers. That’s why I like the idea of Challenge Trials.
Let's say (roughly) there were 30,000 people involved in a traditional trial. So 15k in the placebo group, 15k in the vaccine group (this is roughly the breakdown for Moderna/Pfizer/J&J/AZ). To achieve statistical power, the study designers determined that they would check endpoints when 30, then 60, then 90, then 120 people were infected across BOTH groups (again, roughly). Then they would compare the figures of people in who would be infected and measure the efficacy based on the ratio of people infected. So for example, if 80 people were infected in placebo group and 8 in the vaccine group, that's 90% efficacy (again, these were roughly the numbers you saw in Pfizer/Moderna).
Now if you ask me whether you're putting those 80 people at risk -- yes, you are! But now let's say you run a challenge trial and determine you need 10,000 people to achieve statistical power (I'm completely pulling that figure out of thin air... I don't know the actual number you'd need). So you expose 10,000 people to the disease, 1,000 get infected -- 90% efficacy. You've proven the same thing but exposed many more people to the disease. Obviously the numbers are arbitrary here... but are we really going to be able to get a safety readout amongst a cross-section of society with 100 people? Probably not. What about 1,000 people? Unlikely, but let's say for the sake of argument that you can -- that's still 100 people infected and you're greater than your placebo group in the non-challenge trial!
I just think that thinking about this as a trolley problem is getting a little too cute. You need a real world trial design that addresses these issues. It's not enough to say you're harming people passively versus actively -- I just don't see how a challenge trial reduces the harm at all.
I don’t see how you ever can expose fewer people to the virus in non Challenge scenario.
Suppose, as you say, you need ~120 placebo people to have contracted the virus before you’ve got yourself a meaningful sample. At this point you assume ~120 vaccine people also were exposed and so you’re saying you need an exposure group of ~240 people to make any meaningful determination.
You have two ways of exposing these people:
- expose each in an isolated lab (Challenge Trial)
- tell each to walk around NY until you’ve reached that 120 number
Practically by definition, the latter one is going to require that more than 240 people were exposed! By the time you’ve hit your 120 mark, thousands of New Yorkers would needed to have it in order for it to jump to those 240 people.
The numbers factor out. Call 120 N. For any N (short of some weird edge case where N is the size of the planet’s population) you’ll always limit exposure by doing the exposures in a lab rather than doing the exposures in the wild and waiting and hoping.
If you pick an N of 10,000 you will still require less global exposure by exposing each individually than if you sit around and wait for them to be naturally exposed by the rest of the world. Again- because for them to be naturally exposed millions of non-participants will need to get it.
You raise an interesting point about safety vs efficacy that I hadn’t considered. I suppose the goal there is to vaccinate enough people that you understand the dangerous side effects are under some low bar.
I don’t think this presents any categorically new problem in a Challenge vs a Non Challenge:
you can still choose to vaccinate as many as you want/need and you pick some lower subset of the group to expose.
The vaccinated but non-exposed people give you a safety readout. The vaccinated but exposed people give you an efficacy readout. You cross both with a placebo group so you have a baseline for both.
But does any of that change the fact that you’re still at the end of the day being more _efficient_ about exposure (thereby exposing fewer total people)?
Well now you have two arms to your trial, a safety arm and an efficacy arm. And you've complicated matters immensely. Is this going to provide a cleaner statistical readout than either a straight challenge trial or a traditional trial? And is it going to get you to your answer faster?
Again, Pfizer and Moderna both managed to produce safe vaccines that were 95% efficacious in 10 months, which beat the previous record for a vaccine by over three years. They were so fast that they couldn't even ramp up manufacturing to have enough doses ready by the time of approval (pharma thinks about this in great detail, and Pfizer's CEO Albert Bourla was very clear in the summer that they had already started manufacturing at risk). The onus is really, really high for designing a challenge trial that gets you where you need to be faster.
yeah I've heard the argument that essentially boils down to "well COVID may be especially bad so we shouldn't do it" but like, COVID being potentially really bad is a great argument for why we should do it
Just a quick thought (PhD epidemiologist who studies ID): covid-19 is rather an unusually attractive candidate for challenge trials. You can find me musing about the use of challenge trials if you look closely enough. But it is a disease family for which vaccines had been developed (in animals but that's not nothing), for which there was a lot of research on vaccine targets because of SARS-1, and which had a relatively low fatality rate among young people.
Replace it with either smallpox or HIV and the arguments get a lot trickier.
From a cost benefit standpoint you also have to consider the manufacturing time to produce vaccines at a mass level. As far as I can tell, the longer period for a traditional phase three trial did not delay the manufacturing of either of the available vaccines. It's clear that approving either in August would not have delivered one hundred million shots in arms by the end of the year.
There is a whole separate problem of whether Operation Warp Speed was actually aggressive enough in ramping up manufacturing capacity. And the clear failure to prepare a robust enough delivery system.
Speaking as a fellow philosophy major (and one who's generally skeptical of a lot of what passes for bioethics), I'm not sure this article really engages with the ethical worry posed by challenge trials.
To be scientifically rigorous, as you acknowledge, a challenge trial would have to deliberately expose vulnerable, high-risk individuals to a virus with a substantial chance of killing them. That's different in kind from simply failing to use all means necessary to keep such people from getting infected (we never do everything possible to keep people from being infected, sometimes because of ethical constraints and sometimes because the measures would be too costly). Or to make the same point another way, people in the placebo arm of a normal vaccine trial are no worse off than non-participants in the trial; people in the placebo arm of a challenge vaccine trial will--reasonably often!--end up far worse off than had they ignored the invitation and chosen not to volunteer.
The response to this is, well, they consent to self-sacrifice. But we get a lot more dubious about consent when it comes to serious self-harm (see: hard drugs, dueling). It's really not like the medical personnel case, where there is rightly a huge amount of investment in trying to *keep* such personnel from being exposed to the virus--exactly what you can't do in a challenge trial.
There was a lot of effort to keep medical personnel from being infected, but when the PPE shortage happened, we didn't tell them to stay home. We accepted that they took a very substantial risk of infection in exchange for the benefit we would get from it. The social benefit of a challenge trial participant are way larger than the benefit of a nurse showing up to work in a cloth mask.
There are plenty of other better takes than mine on why we allow (and celebrate) people taking all kinds of risks. Medical professionals, military, law enforcement, astronauts, sports people, etc.
For fun though I'd like to take on the harder case... "we get a lot more dubious about consent when it comes to serious self-harm (see: hard drugs, dueling)"
Should we though? The (welcome) trend towards drug decriminalisation is explicitly recognising that people can take decisions which may be harmful. That's currently confined to 'soft' drugs but personally I'm in favour of it being extended to hard drugs.
In a similar vein look at combat sports (boxing, mma, etc), there's no question that they involve a high likelihood of serious harm and even death. Yet in large parts of the developed world we permit consenting adults to compete in combat sports. And again I personally see no ethical reason that couldn't be extended to dueling.
If you recognise an individuals sovereignty over their own body then the only real room for ethical objection is to say that people are incapable of making a valid decision because they:
1. have a mental disability/illness
2. lack information
3. are subject to some kind of coercion (actual coercion or much looser concepts like poverty)
4. are in some kind of catch-22 (only sane people are allowed to consent to being killed, only an insane person would consent to being killed).
There’s no social benefit to hard drugs or dueling. If you’re going to chide people for not considering the ethical issues you should be careful not to stack the deck by disregarding the benefits.
I mean, I guess you could try to select for the people who would seem likely to get it anyway, but if you're really good at that you could use that approach to reduce the time of the normal Phase III too. Might be pretty hard to get people >65 that way.
You actually can't do that! That proposal makes no sense at all! Transmission in nursing homes is a massive driver of deaths, it's vitally important to know the vaccine works in elderly people. (My recollection is that Trump had to threaten to fire the FDA head to get an EUA shortly before it would have issued anyway in a context where the logistics of vaccine distribution to states, let alone into people's arms, were not meaningfully sped up at all--appearances over substance as always with them.)
The problem with this is that for every member of the placebo group who gets COVID in a non-challenge setting, there are thousands in the placebo group who do not.
In the challenge placebo group, there could very well be a 100% chance of infection.
I'm still utilitarian enough to be vaguely in favor of challenge trials, but I don't think this particular argument really holds up.
I don't know the right answer to the main question here (sc. "Challenge Trials: yea or nay?"), but I have a comment about the dynamics of the debate.
I suspect that people react to the choice between challenge trials and standard Phase 3 trials as though they are facing a choice between killing and letting die.
Some people think that distinction (i.e. causing harm vs letting harms occur) is ethically irrelevant, and indeed a source of ethical confusion. (This group is not perfectly coextensive with consequentialists, but fairly close).
Other people think that it is a distinction of fundamental ethical importance, which changes actions from permissible to impermissible: there are some things that one may not do, but may allow to happen. (This group is not perfectly coextensive with deontologists, but fairly close).
I believe the research on popular belief among non-philosophers shows quite a lot of support, more than majority, for the ethical importance of the killing/letting die distinction, and the causing/allowing distinction in general.
So you start your campaign of persuasion with some hard work ahead of you. It runs counter, or at least seems to run counter, to widespread and deeply held ethical intuitions.
It's "taking a risk of dying with consent." Do people who oppose challenge trials on ethical grounds also object to people donating a kidney or half their liver? If not, why not? I don't know the precise odds of dying or having a nasty outcome from that kind of organ donation, but I doubt it's on a different order of magnitude from the odds of a young person dying from covid.
Agreed, Alexandre. Those are factors to stress in the campaign of persuasion. They help to shift the frame of discussion away from one way of looking at it, towards others.
(MY already does this in emphasizing voluntary heroism).
And popular ethical reactions are often highly sensitive to framing effects.
This is a sort of inchoate comment on my part, so I apologize if it's hard to follow my reasoning.
I feel like there *is* a difference between killing and letting die, and that on some level consequentialism is not moral. But simultaneously that the government or public policy, dealing as it does with vast swathes of people, is inherently immoral in this manner. We do not live in a world in which societal policy (whether mediated through the government or some other institution) can be tailored to the unique situation of each person affected by that policy, and so... you've got to be consequentialist about public policy in a way that you don't and maybe shouldn't about some kind of individual action. The trolley of the thought experiment is different in kind to collective action.
Right - the consequentialist argument here is not necessarily "consequentialism is right", so much as "one side of the outcomes is so much more consequentially worse that consequentialism swaps other concerns."
If the number of people who get sick and/or die in the trials, in total, is the same, I think you'd have a nearly impossible time making the case for challenge trials. It's the harm to the rest of the community while you run the small trial that drives this discussion. And Matt's argument is that that harm is baked in, because if the larger community isn't having a dumpster fire like the US, the phase 3 will take longer and be harmful by duration.
I'm sympathetic, bsupnik, but "the consequences are so much worse that they swamp other ethical concerns" just *is* a claim that consequentialism is right.
Which is fine! That's the claim consequentialists should make! (And argue for, ideally.)
But you can't avoid it by saying, "I'm not saying cons. is right, I'm just saying those considerations override all other ethical concerns."
Ha ha ha ha I came so close to sneaking the conclusion into the premise. ;-)
I guess my thinking was more like "consequentialists think consequences matter and therefore may be able to make a relative moral ranking based on the measurable scope of consequences. So it's easier to argue against a consequentialist (after you accept their premise) if they are forced to admit their consequential moral differential is small."
You're right that this would be conceding the moral framework, but if you're not going to do that, you're not going to have an argument, you're going to talk past each other anyway.
Yeah, that makes sense, and I bet a lot of people would agree.
Your rationale for it seems to blend together two different distinctions, though:
1) state vs private
2) large numbers vs small.
On the first rationale, states may employ consequentialist reasoning that private individuals may not -- on matters both large and small. The state may kill a thousand to save a million, or kill one to save five, even if that small-scale choice will never have larger consequences. Private persons may never do either, but states have a different moral status.
On the second, both states *and* individuals may kill a thousand to save a million, but nether states nor individuals may kill one to save five -- the numbers are just too all.
You could also have the view that both stipulations are necessary: no consequentialist reasoning unless you are 1) a state dealing with 2) very large numbers.
I think I have two different axes of thought here:
1. I am not sure that states, or other large organizations, should obey the same deontological rules that individuals do.
2. I also think that the error bars on consequentialist decisions are HUGE. And that outside of idealized thought experiments, even if you buy into consequentialism entirely, you have to deal with that. Which makes a "kill 1, save 5" thing problematic, but a "kill 1, save 10,000" thing much less so.
There are many questions about how public health officials evaluated costs and benefits to come to the decisions they mad. Why no challenge trials? is just one:
. Why no mass testing of asymptomatic people?
. Why no compensated self-isolation for asymptomatic people who test positive?
. Why the delay in approving cheap, fast screening tests?
. Why were recommendation of different kinds of social distancing, mask wearing, ventilation, traffic flow in different kinds of venues not tested to provide feedback on best/least cost practice?
. Why did officials not say for the beginning that recommendations would change as they learned more (not wearing/wearing masks, importance of hand washing/scrubbing surfaces)?
. Why the virtual silence on the point that social distancing is as much to protect others as to protect oneself. Why "Stay safe" instead of "Be safe"?
. Why was messaging not based on explicit, evolving models with recognition of how models were changing.
. If there are good answers to these questions, why have officials not supplied them?
The general impression is of total improvisation, as if the very idea of a pandemic was totally novel concept in public health.
Challenge trials are only more ethical in a very academic sense, which is why I think you're drawn to the topic. Obviously, killing people in an experiment vs people dying in an experiment are two very different things. And it's easier to talk about in the context of a low fatality disease like Covid, but what if the disease in question is overwhelmingly fatal? I guess you could get comfortable with killing people intentionally from a strictly utilitarian perspective, but I don't think that's where society's ethical code is at this point.
I think society doesn't support the strict mathematical utilitarianism that some people want to do. Kill one person to save two. Kill three to save four. You're going to have a tough time getting people to agree with those proposals, and, I think, correctly so.
But I do think that there comes a point where it becomes much harder to make the case against utilitarianism. Let's say we did challenge trials with 10,000 volunteers, and that resulted in moving up our entire vaccine timeline by 2 months.
That's plausibly something that kills less than 15 people (5,000 get the vaccine and of those maybe 125 get infected and due to milder infections none die, 5,000 get the placebo and 2,500 get infected, 12 die), and saves 150,000 lives. Each death saves 12,000 people. I think that kind of ratio washes away a lot of the ethical concerns we would have about killing one to save two.
I agree that the covid numbers make a pretty compelling case, but I still don’t see societal acceptance for intentional killing in the name of science. And when we look at more dangerous diseases it strikes me as a real non starter.
When you say killing people, the image that comes to mind is a scientist controlling a drone, injecting random people without their knowledge and seeing what happens. Overly dramatic, I know, but the point is you are overlooking the critical point of *they would be volunteers.* People do choose, regularly and of their own volition, to do something for the greater good even at great personal risk. People do this all the time. There are probably a dozen professions that in total make up a large percentage of the work force that would constitute a risky profession that isn't entirely justified by the pay. But obviously people do them anyway because they *want to help.*
Matt's comment towards the end about medical ethicists taking a dim view of self-sacrifice is spot on. It is a serious weakness in thinking. People like to do great things, and ethicists should let them - especially when the alternative is potentially tens of thousands of deaths.
Again, covid makes this an easier conversation. But let's say it's ebola, which is 90% fatal and it's a horrible death. I really don't believe there would be a long line of right thinking volunteers no matter what the math is, ie how many people my death would save. The people who would volunteer would overwhelmingly be suicidal and therefore not deemed capable of making this decision. Or you could open it up to people "volunteering" for large sums of money, release from prison etc but that is a whole other ethical can of worms. Probably the cleanest group to deal with is people with already fatal diseases with only a few months to live, but they would present obvious issues with the results. These professions you talk about are things like police I guess, where their risk of death is higher. But they also enter into the profession assuming they won't be killed, due to good luck or higher skill levels. And they are almost certainly right, because the probability of death is tiny. People entering an ebola vaccine trial will be virtually certain of death. I just don't see it as a valid comparison.
People might surprise you. Firefighting is majority volunteer position, for example. Extended periods of boredom punctuated by running in to a burning buildings sounds on paper like a really terrible idea, but here we are.
But in any case, I think something like ebola (with a very likely chance of death) might just a logistical exception and not a proof of a failure of ethics. Covid isn't ebola, and there would've been no lack of volunteers.
Well, if you can't get enough volunteers for a challenge trial, then, okay. No challenge trial.
But if we're in a counterfactual world where there's some horrifying ebola epidemic/pandemic, such that people are seeing their friends drop like flies to ebola, I think you might find a surprising number of people would be willing to do it. I might be wrong! Hopefully we never find out!
The scene playing in my mind is Armageddon, where the group of heroes know they stand a good chance of dying to save the planet, and they get "1 free wish + never ever have to pay taxes again" as the reward for their heroism.
How is a deadly pandemic not different than an asteroid on a collision course with Earth?
Good thing to ponder, especially given that at the start of COVID, with some of the scarier long haul stuff happening, we really didn't know if this would be much worse.
It seems like there would be more societal acceptance if experts were saying “actually this could save a lot of lives but it does raise certain ethical questions” rather than “This is a really bad awful idea with no merit whatsoever.”
…
I’m sympathetic to your argument here, but a lot of the anti-challenge arguments are very weak because they refuse to make the concession you do that “yes, it could very well save lives” that quickly makes folks like Matt and myself frustrated because it appears the medical community is being dismissive for condescending credentialist reasons.
Also slippery slope reasons—like comparisons to Tuskegee and Dr. Mengele get brought in a lot, even though they’re not at all apposite to a voluntary trial.
There's an outstanding question of whether we can be confident that people can meaningfully give informed consent for these trials, which does have some Tuskegee-like properties.
That's one element of this that I don't think consequentialist philosophy has much to say about.
How on earth is it "Tuskegee-like"? The Tuskegee subjects were simply lied to; they were told they were getting health care when instead they were injected with a disease. Whether or not it's possible to give informed consent for a virus trial—and I'm not sure why it's not—it's not within a hundred miles of giving people a disease without their knowledge or consent.
Having people engage in a risky behaviour, where the risks and population involved are high enough that deaths are probable is something we regularly do in the name of science.
People are much more selfish then you give them credit for. Alot of people will happily let the volunteers die so that they and their loved ones don't suffer. After its over they'll mouth platitudes about how brave the sacrificed were and go back to voting against any tax increases to improve healthcare for poor people.
Was thinking... we knew the Apollo missions had a substantial chance of death, yet we did those in the name of science. I wonder how this is materially different, other than perhaps the scale.
Frankly I think a more deadly disease may make the idea even more popular. Take the same numbers as Michael uses with a IFR of 10%. That same trial would kill a lot more people (something like 260 people). But of course if the same number of people in the general population were infected with that fatality rate you'd have a lot more deaths, and the lives saved per death in the trial would be roughly equal.
Of course rough utilitarian calculations like that do not make for popular ideas, but with a virus that deadly, societies views on sacrifice would be WILDLY different. I think telling people we want to put some of you at great personal risk (something like 5% chance of death) for the sake of saving us all, in the context of millions of people already dying, is not an idea people with fear in the desperation and loss of life around them. Getting volunteers may be hard, but selling it to the public seems quite possible.
I'm a microbiologist by trade, and I think that it was a huge mistake to not to human challenge trials. Among other things, it's much easier to determine whether the vaccine induces sterilizing immunity, ie if people are protected from getting infected. And if they are infected, do they have infectious virus? It's also an efficient way of identifying which vaccine candidates are likely to be more or less effective. Still, human challenge trials are not enough and we'd still need phase 3 trials to detect rare events. The high fatality rate for older people makes human challenge studies that include people over 60 a harder question. (Then again, if a vaccine is >95% effective for people 18-55, it's almost certainly going to be pretty effective for people 65+.) But there are some challenges for human challenge trials:
We need to optimize infection: what is the inoculation dose, in other words, how many "live" viruses do we use? How do we infect them? Aerosols, droplets? Do we expose them to a high dose over a short period of time or a lower dose over a longer period of time? The world of immunology is full of infection models that are very reliable, ie the animal is reliably infected, but the infection method is somewhat different from how natural infection occurs, so it is only somewhat applicable to real-world infection.
This means that we'd have to spend some time optimizing the protocols before testing any vaccines. It's going to take quite a few volunteers.
If you run into trouble optimizing human challenge trials, phase 3 trials may end up being faster. But with a pandemic like this, you need to throw the kitchen sink at it.
I do think it's important to make sure that the volunteers are not pressured to take part and truly understand what they're doing. It's also important to remember that some subjects might die, and so you have to prepare the public for it.
I agree that we should do challenge trials, but this seems to almost entirely gloss over the actual hurdles to doing them:
1. Regulatory uncertainty. The FDA website says that companies need to discuss human challenge trials with the FDA before they are conducted. If that process is slow, then any potential time savings are wasted.
2. Public reaction. I think that the challenge trials are ethical, but how will the public react if someone died in a Pfizer challenge trial? Does Pfizer want to risk their reputation on that?
3. They are scientifically inferior because they don't test under real world conditions.
But yeah, my sense is people decided it was a bad trade-off (which you can disagree with) but then used ethics as the convenient, easy-to-explain reason to give on Twitter.
I happen to think that the biggest relevant objection is the epistemological objection 3 that Sam mentions in the other comment. But if one doesn't think that's such a problem, then 1 can be addressed, by an FDA planning committee deciding *this year* what sorts of conditions and regulations for human challenge trials would be allowed in *future* pandemics. (Presumably it would depend a lot on the nature of the risks of the relevant infection - we're probably more happy doing this with infections that pose a major risk for a short period that a good fraction of people recover from than with infections like HIV in the 1980s that pose a risk for a long period that people never recover from.)
Another way of saying this, if your phase 3 design means you know people are going to get infected, just over time, because they took the placebo, isn't it just more of a "passive aggressive challenge trial"?
One challenges via natural infection in society at large, the other does so under supervised medical conditions. If the difference is challenge can be faster, especially for things with an IFR under, say, 2-3% where you are using what appears to be a pretty safe technology in mRNA... it seems unethical to *not* do a challenge trial.
Downthread one objection is "but need to re-create natural infection conditions".
I would submit that a good study happening in phase 1/2 that aggressively works to find this out would be a good way to:
a.) Allow for challenge trials that "challenges" in a way that simulates natural infection
b.) More quickly understand the virus attack vector so we don't end up, say, doing a panic run on hand sanitizer when it turns out it's not terribly effective at preventing this specific disease
(generally, washing your hands is good, so maybe we got some nice side effect habits out of this... but that's another thread)
I think that some of these issues could be addressed by "challenge trials" of real-world behavior. Find known infected and known uninfected people, and then have them go to a bar together, or grocery shop together, and tell them to change their hand-washing practices, and figure out which routes of infection are most significant. Then, when the vaccine is ready, you also have good models of real-world exposure conditions, so that you're not using an unrealistic challenge of injecting people with the virus.
Again it all seems obvious until you design the experiment. “Just expose people to the virus” does a lot of work for you but it by no means an experimental design. Please interview a scientist on how they would choose to do this experiment and your argument will be stronger. Do you put the test subject in a room with sick people? Inject it? Gas people with it? How large is the exposure? When you do it the standard way, people are exposed to “normal” levels of the virus not some standard that must first be scientifically established. Not all viral infections are the same, quantities and methods matter.
Agree 100%. I'm a biostatistician and the design of the Pfizer/Moderna trials are dead simple, based on decades of clinical experience and statistical research. An undergraduate statistics student could the study design.
For challenge trials, the design is way more complicated. Even determining what results from the trial would mean that it works is extremely hard, because that depends on how contagious the disease is, the exposure setup, and the effectiveness of the vaccine.
The FDA said they would approve any vaccine that is safe and at least 50% effective. I would love for a proponent of challenge trials to describe an experimental design that would assess the effectiveness of a vaccine with a true effectiveness of 50%. It's not a trivial problem!
This is not the attitude that got us to the moon. The specialists should have found a solution to this not-trivial problem. But clearly they felt no responsibility to do so.
Anyone can see that tests were not expedited in the best possible way. It's up to the specialists to figure out what would be a better way. Anyone can identify the problem, only a specialist can pinpoint the ideal solution. Why is that suprising?
Sure, but that takes time. The Pfizer and Moderna trials each lasted about two months after vaccination. Best case scenario, a challenge trial saves a few weeks? A month? Maybe an additional few weeks saved during the enrollment, which would also be smaller/faster.
Obviously, even a month or two faster would save a lot of lives. But any additional time spend researching study design would very quickly eat into those savings. The Pfizer/Moderna trials could be designed in a few hours, so if it takes a month to research and design the challenge trial, it's basically a wash.
I think the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics.
"the best argument for challenge trials is that we should fund research now so we can quickly deploy them for use in future pandemics."
This seems like the place we should all end up converging. Hopefully the people who start thinking about this stuff now will end up coming to some sort of consensus about the ways in which challenge trials will and won't be a good thing in a future pandemic, and can come up with streamlined procedures for expediting the ones that should happen, and clarifying which ones shouldn't happen.
In the mean time, I think there should be a lot of research devoted to ecological challenge trials of respiratory viruses generally. It's only this past year that I finally realized how little we know about the relative role of surfaces, droplets, and aerosols in the transmission of various "cold" viruses and influenza, and about the relative effectiveness of interventions targeted at children or adults, including school closures, social distancing, and mask use.
This is an optimistic vision, but I'm afraid that the consensus you hope for will be forged by hidebound cover-my-ass/"maintain the public trust" FDA types like Eric Topol, Paul Offit, and so on.
>>But any additional time spend researching study design would very quickly eat into those savings.
Couldn't they have done that research during the phase 1 and 2 trials?
Im interviewing to PhD programs right now do you think this is a good gateway to your field or is masters more the standard? Ive been in biomedical research for a while now and I’m wondering if my experience is sufficient already if I decided to go that route. Id like to do basic research regardless just worried about the long game.
If you want to do biomedical research, I'd say a PhD is standard. Quite honestly, a large portion of people who get masters in biostatistics in my department end up in data science jobs in tech.
This seems like a good sort of way to do it, but it seems far from obvious to me that you'd be able to both end up with fewer people infected and a shorter trial time. I guess each round of this trial would take at least a week, to find an infected and infectious person, use them to expose several volunteers, then keep the volunteers isolated until you can be sure they didn't get infected for that interaction. If you start with a large enough number of participants to be sure you'll get significance early, you'll probably be using a larger number of people than the standard Phase 3 trial used (since you're looking at just one afternoon of exposure, rather than the number of exposures that members of the public are averaging); and if you start with smaller populations until you get a sense of the exposure time needed, you'll end up needing to do many rounds (and the standard Phase 3 trials only ended up taking about ten weeks anyway).
These are surmountable study design and methodology questions that come into play once you get past threshold ethical objections to challenge studies at all. At most they come into the ethical discussion only insofar as they might undermine the premise that in a particular case a challenge study is more ethical because it would be faster and thus save lives.
I disagree in that I actually think the experimental design actually drives the ethics. For instance if I have to get a sick patient to cough on my test subject then Im potentially causing another human being to be responsible for the sickness of another who maybe is less qualified to understand the ethics or isn't in a good position to think about the consequences due to illness. I wouldn’t be surprised if there was an ethical way around these concerns but they need to be clearly stated. No amount of hand waving will make this core problem with Matts argument just go away.
That seems like just a routine question of informed consent, which comes up in every study involving human subjects. IRBs are very experienced at handling those. It might be harder to achieve informed consent for some challenge studies but that's not a good reason to categorically take challenge studies off the table.
If I might rephrase: challenge trials are a great way to test the efficacy of a vaccine against deliberate infection, but what we want is a vaccine that works in the real world. And we can't know if the former necessarily implies that latter, particularly with a novel virus.
With all do respect, I am an active, professional, PhD-having research scientist and I don't agree with that. The only way to establish that efficacy from deliberate infection maps directly on to real-world efficacy is to compare the two in rigorous studies, but in the absence of challenge trials, that study is not possible. I think Matt is out of his depths here because running challenge trials *now* would violate fundamental principles of the scientific method; at some point in the future they may become useful, but barring some huge cache of literature comparing the different methods, we're stuck with what we have for now.
Granted, my expertise is (bio)chemistry, so I am not an expert on infectious disease, but there are nuances to the method, frequency and level of exposure for many known viruses that are not strictly multiplicative. I see no reason that would not be true of the novel coronavirus. You would need to design a deliberate infection model that covers a sufficient combination of those (and other) relevant variables to be able to assume that it is sufficiently simulates real-world infection because every downstream assumption would inherent the flaws in the deliberate infection method. You would also need a sufficiently large group of humans to cover the natural variance in human immune responses, co-morbidities, etc.
It is entirely possible that I am entirely wrong, since the devil is usually in the details and I am not an expert. But at first blush, this is simply not how science works.
The way trials are currently designed, the statement that "there was an X% decrease in rates of infection with Y certainty compared to a placebo" is accurate. If you deliberately infect people as part of the trial, the statement narrows to "there was an X% decrease in rates of infection in people deliberately exposed to the virus with Y certainty compared to a placebo". Moreover, both groups would need to be held in controlled conditions for the duration of the study, or you would have to narrow the statement further to include the caveat that you did not control for random exposure.
"The relationship between vaccine efficacy and infection efficacy on success rate is *strictly multiplicative*. A vaccine that reduces prevents 63% of infections against a 70% infection rate is just as efficacious as one that prevents 81% of infections against a 90% infection rate: 90% effective."
I think this is technically correct, but not actually informative. We *define* vaccine efficacy as the ratio of number of infected with the vaccine to number of infected without the vaccine, in two populations with the same other characteristics (including distribution of exposure). But the efficacy is an efficacy *for that population*, and not in general.
There's no particularly good reason to think that a vaccine that has an observed 90% effectiveness in a population that is constantly kissing each other would have the same observed effectiveness in a population that is sharing needles, or a population that is grocery shopping with masks. If the vaccine works by creating a certain level of immune response, it could be that this immune response is 100% effective against viral doses that are below a certain size, but not against doses that are above that size (if they manage to overwhelm the response and infect many cells before the body gets it under control), so that it is much more effective for the grocery store population than for the IV drug population. It could be that the immune response is concentrated in the lymph fluid rather than the bloodstream. It could be that the immune response prevents a mechanism the virus uses to infect blood cells but not lung cells, so that it's more effective for the IV drug users than for the grocery shoppers.
Another way to put this is that in addition to whatever ethical problems they may or may not have, challenge trials have epistemological problems. They give you knowledge about how the vaccine does or doesn't protect against certain styles of lab-based intentional exposure, which doesn't give you much clear sense of whether the vaccine does or doesn't protect against the real-world kinds of exposure people actually get.
However, a friend pointed out to me that once we are considering challenge trials, there's room for them in a *lot* more than just the vaccine side. Right now we are doing all sorts of NPIs (non-pharmaceutical interventions), which are the medical euphemism for things like closing bars, telling people to wear masks and stay outside, and so on, but we have basically zero data as to whether or not these interventions are effective, and if so, how effective.
Once it was clear that this virus was not very harmful to 20-somethings, it seems that it would have been a good idea to do some challenge trials to measure these things. Get a few hundred volunteers to hang out in a bar for science, and do testing first to ensure that most of them tested negative, and a few tested positive, but don't tell them who is who in advance, and see who tested positive afterwards. Then do the same thing, while telling everyone to stay six feet apart, and requiring the bartenders to wear masks. Do the same thing with concerts and grocery stores and meat-packing factories and playgrounds and such.
If we had started this stuff in April, then maybe by September we would have had a better sense of how likely various types of exposure are to spread the virus, and how much different various interventions make to both individual and social risk. It might also have told us which social protocols are particularly effective ones at spreading, which we could then use in vaccine trials a month or two later, rather than relying on unrealistic injection scenarios.
Unfortunately, the closest I found was this:
https://www.npr.org/sections/coronavirus-live-updates/2020/08/24/905534790/german-experiment-tests-how-coronavirus-spreads-at-a-concert
Of course, one thing that understandably raises huge ethical concerns about this is that this is basically the study design of the Tuskegee syphilis experiment - get a bunch of people, test them, don't tell them the results, and then see how the disease progresses and spreads to others when untreated. But if you are working with a known low-risk population, and give everyone information a week after the experiment, and good treatment, and informed consent before the trial, you can mitigate a lot of that.
This seems like a minor technical issue (mainly because if the vaccine works, it will work against any method of exposure, and vice versa). I guess just try to emulate some known superspreader event for each participant?
I think one natural experiment one could do would be to compare the rates of death of healthcare workers to that of the average population. My hypothesis is that they die at higher relative rates than their age cohorts. If so this could demonstrate the way in which heightened viral exposure increases the mortality rate. I suggest this because it would show that a higher environmental dose results in more deaths. The point is we would need to settle on what is a typical dose this all takes time as well. Totally get what you are saying but this is probably how you would have to get at the question of what dosage emulates a super spreader event. I think it’s possible but may be a bit trickier than most imagine.
Further complicating lab infection: there is precedence for the sensitizing effects of a vaccine creating potentially worse outcome in a lab exposure than in wild exposure. The truly informed consent becomes harder because a subject could reasonably expect that the lab infection would mirror a real world infection when it could be much worse not only because of the issues you raise (strain, load, route) but because of the vaccine itself.
This is a great point. Even with COVID19 after all these months, there is considerable uncertainty about the "viral dose" someone receives out in the world and how sick they are likely to get. It seems to be mediated by masks, where masks reduce the dose and thus result in a mild infection. But, there is also quite a bit of variation in how strongly someone's immune system will react to infection. For a truly novel pathogen where the route of infection is not thoroughly understood, it would be difficult to design the challenge trial.
I think that trying all the methods in different groups is probably a good idea, but then you get back to needing large study populations.
I’m all for challenge studies to get a quick read on efficacy but I don’t see that doing anything about Phase 3 studies. Vaccine Phase 3 studies are not the size they are simply because of efficacy. It’s because you need large trials to identify safety signals that are too rare to pop up in a small Phase 1 trial. Covid is bad but most people haven’t had it and the vast majority of people survive. The vaccine is going to go into *everyone on the planet* in an ideal world. The risk/benefit ratio is hugely critical for a scenario like this, and it’s very challenging to track adverse events in a non-controlled environment. To avoid Ph3 studies you would need a new way to accurately track safety in the general public.
You can separate out the need to do Phase 3s to test efficacy and the need to test safety and just run big safety trials. Here's a proposal along these lines from early in the pandemic related to challenge trials. -- https://www.medrxiv.org/content/10.1101/2020.05.18.20106187v2
This argument is much more logical than the "challenge studies are inherently unethical" response, which is another flavor of the same general story of medical ethicists being totally divorced from the actual study of regular ethics.
Right. We don't talk about this much - maybe because there's the crazier "vaccines cause autism" meme out there - but if you give an intervention to a young population planet-wide, the side effects have to be really close to zero to make it not a problem.
I think this is also a risk we eat no matter what to some level - the youth of today are the longevity test group.
At first I recoiled, but you convinced me on the merits. We have an all volunteer army risking their lives to protect the rest of us and that doesn't seem unethical. At sporting events we could cheer volunteers who risked their lives in challenge trials and offer our gratitude and collective sorrow at funerals for those given placebos. Seems odd, but better than the alternative of greater death.
We also allow people to undergo abdominal surgery and donate one of their kidneys or part of liver. This is a medical procedure with no benefit to the donator that carries some risk of bad outcomes including death. It seems like a very good analogy to me.
>>We let people volunteer to get shot at for the common good, and as bad as COVID may be, for most people it's a lot less risky than getting shot at.
Yes, but... That shouldn't be the crucial point here. Because it's even more crucial that we be willing to use these methods with deadlier diseases. With something like the MEV-1 virus from Contagion, slowing down trials would mean killing millions.
It’s the trolley problem!
My question: do challenge trials exist in other countries that emphasize the collective over the individual? I am sure an apples-to-apples comparison is challenging considering situations like the suspect trials of Russia’s Sputnik vaccine.
Well, it's a variant of the trolley problem where you get to yell to the person that you're considering switching the track to, "Hey, are you okay with this," and he yells, "Yes!"
Well put!
Well, no, that person is going to be very very very very likely to be sick and not dead, but there is a very small risk that that person will be dead.
I think the utilitarians are very comfortable with that.
I have a pet theory that humans round all risks up to 100% or down to 0%. Maybe this creates a problem for informed consent? I'd still let the volunteers take the risk, backed by what medical support we can give them.
Saying that people round risks up or down seems appealing for a lot of examples, but it doesn't explain the weird way that people overestimate risks from airplanes and bike riding and underestimate risks from cars and infectious disease. I think familiarity is also said to play a major role in people's estimate of risk, as well as whether the risk is thought to be something that the individual controls or is controlled by someone else.
Totally spot on. I work in aviation more or less, and we get absolutely crazy when non-aviation people get cranky about the risk. The indignation usually goes something like "how can you say that and get in a car??!?!"
Right, you have to use inactivated or attenuated trolley-cars.
Two limitations of the challenge model that challenge trial proponents rarely if ever address.
1. What is needed is a vaccine that prevents infection from real-world exposures to the virus. A challenge trial exposes patients to the virus in a different way, everyone getting a set amount via the same type of exposure. So a challenge trial should be great for clinical proof of concept, measured efficacy in a challenge trial may differ from real-world effectiveness of the same vaccine. Expose people to not enough virus and you may overestimate real-world effectiveness; epose them to too much and you may understimate vaccine potential and even reject an effective vaccine.
2. Challenge trials would be smaller and faster, which means conserably less safety data.
Both Moderna and Pfizer vaccines essentially charged right into Phase 3 without the typically slower process of first showing some efficacy in Phase 2, helped by the government's guaranteed purchase. Either or both of these vaccines could have failed in Phase 3, as many drugs do. Challenge trials would offer the advantage of weeding out ineffective drugs quickly.
I have nothing to add here; I just want to know the most plausible way to make challenge trials happen. I think the best strategy might be persuading the government of Russia or China to do them.
The worry about this is that if Russia or China do something that is judged to be as morally bad as the Tuskegee syphilis experiment, or the Nazi medical trials, then they could set back the whole cause of getting challenge trials approved. You want to be able to show people that it is possible to do challenge trials in a way that appropriately takes the ethical concerns into account, not just show that challenge trials are possible.
I don't think anyone has proposed doing challenge trials on people other than consenting adults. Matt had the patience to write a lot of words but TL;DR: it's frankly bizarre that letting consenting adults put their health at risk for the good of humanity is considered ethically dubious.
The problem with doing it in China, for example, is making sure that all the volunteers actually volunteered. I don't just mean to impugn China here, but also to note that the international press does not presume good will on the behalf of the Chinese and might end up concluding (whether correctly or not) that people were forced to take part.
I lived in China for more than a decade. It may be a police state but it's not a forced-labor camp (okay, maybe parts of Xinjiang). If the international press wants to find out whether subjects in clinical trials are participating of their own free will, they should interview them and see what they say. To make that possible the government would need to be more transparent than it normally prefers, but it's not a heavy lift.
Yes, I agree... part of my point is that to some extent the Western press is biased against China, and that could lead to problems with the Western public perception of trials done in China.
Matt, may I arrange to have this piece translated?
Sigh. No. The main ethical reason for not doing challenge trials is that COVID-19 is a new disease whose long term effects are unknown. Back in March or April, there was essentially no effective treatment; now at least, hospitals have figured out how to reduce the fatality rate somewhat (assuming they're not overrun with patients). But this is not a disease you want to have!
Rather than rant for several paragraphs, I'll defer to this Derek Lowe post, who has laid it out far more clearly than I ever could: https://blogs.sciencemag.org/pipeline/archives/2020/07/02/challenge-testing
Is the sighing necessary? This “why are you asking such a dumb question” tone has been the dominant one from challenge trial opponents in the discussion I’ve seen and it smacks of “I was taught this was not an option years ago and I’m not confident in my ability to explain why.”
Sorry... the sigh is because this argument has been refuted by others over and over and it's frustrating to have to keep at it.
To be clear, because I guess I haven't been here: I don't think challenge trials are *never* an option, just that (a) the uncertainty around the science with COVID suggests that they would've been a bad idea here, and (b) challenge trial proponents need to be a lot more clear in how the study design lends itself to faster vaccinations. The details matter, a lot.
It hasn't been "refuted"; the arguments have been aired and they are not convincing to lots of people (including me).
I admire you for standing for the opposing position and taking the heat from Matt's fanbase but your position here is completely unconvincing to me.
The status quo got alot of people killed or very very sick and has made those of us with high risk loved ones live in constant fear for almost a year now.
IMO public health officials need to do a better job of explaining why allowing all that death and suffering was better than deliberately infecting volunteers, because I find Matt's argument very compelling. I'd be willing to volunteer if it meant my loved ones got a vaccine quickly and would not get sick especially if government gaurenteed good healthcare and death benefits in the event that I got sick or died.
The *status quo* didn't get a lot of people sick; deliberate choices by political leaders to achieve "natural herd immunity" did. Matt did a good job laying out the NPIs and I don't need to discuss the politics of that side of it.
The problem that challenge trial defenders need to answer, as I've stated below, is: what sort of trial design gets you to safe, efficacious doses given to enough people to achieve herd immunity the fastest? Pfizer and Moderna each produced a 95% efficacious vaccine in 10 months! That's three years faster than any previous vaccine. They did it so quickly that manufacturing hasn't been able to keep up with demand, even though they both ramped up manufacturing this summer. So where would the time savings come into play with a challenge trial?
The time savings are abundantly clear. Even if only 3% of the nation had been slowly administered in September/October/November that would have saved a huge number of lives both directly and a lowering of the R0 before the weather got cold.
Every single political argument we are having now about speed/priority is peanuts compared to the huge amount of lives that would have been saved if the system was designed for time-sensitive pandemics.
Sorry, the refutation isn't very convincing. Millions of people have been infected with the coronavirus. Infecting a few thousand on purpose to validate a vaccine seems like a no brainer. And it doesn't need to be 50-50 vaccine and placebo.
I don't buy Lowe's argument. The core of his argument, as you say, was that the virus may have been really bad. If anything, this only boosts the Yglesias angle.
The question boils down to: is our test population the full globe? Or is our test population those volunteers who sign up?
You seem to be saying you'd rather the test population be the full globe because... the virus might be really bad?
I think there's some ethical view which says: we're not responsible for the deaths of 400,000 because those are "natural" whereas we would be directly responsible for the deaths of those 100 volunteers who signed up and got placebo'd and died.
Some people are comparing this to the trolley scenario, but the key distinction is that those 100 dead people are volunteers, who presumably understand the risks. Now the burden falls to you: why are you blocking volunteers from risking their life to save 399,900 people? Are those deaths now "unnatural"? Or are they now directly on the hands of the people who made this miscalculation?
A lot of people in this thread have made a similar argument so I'll try to respond to that here. I'd argue that 400k deaths (in the US alone, not globally) represent a massive policy failure. Or even more explicitly, it was a policy *preference* of those in power (the "let's achieve natural herd immunity" philosophy). I don't think you can argue that a challenge trial would have prevented those deaths unless you can argue that government officials would've been more prone to provide restaurant relief, enforcd mask mandates, increase unemployment insurance, etc. with an earlier vaccine (because they clearly haven't been willing to do so up until now).
More importantly though, the ethical angle looks a lot different in spring/early summer 2020, when such a decision would've been fraught with more unknowns. There simply wasn't the data on the nature of COVID (how to treat it, long term effects, who's more prone to infection, etc.). And again, we know that even people who recover are prone to heart problems and other long term effects. The nature of a challenge trial is philosophically similar to "right to try" advocates for drug trials (which is a larger discussion, but it can be argued that this could lead pharma to turn into the herbal supplements industry). You can tell people that these are the risks, but the picture looks a lot different when it's 3-5 years later (or more) and you're still not fully recovered.
I'd also argue that the "run a challenge trial" proponents need to think carefully about study design. How many people are you running this with? How are you exposing them to the virus? Masked or unmasked? What's the viral load? What's the strain? How long are you exposing them? How are you preventing them from being exposed outside of the confines of the study? How would we measure vaccine safety on a smaller population? A proper challenge trial study would need to answer all these questions and argue that getting to an answer would happen faster than the incredibly fast development we've seen with the Pfizer and Moderna vaccines (which met their targets of a Phase 3 readout in mid-fall 2020).
I think everyone agrees that the actual harm of Trump administration policies is (1) really bad and (2) a total failure for which we could have done better.
But I think the argument for challenge trials isn't that "we need them because we suck at mitigation", it's that the non-challenge trial needs a certain amount of naturally occurring harm to happen to the placebo group in order to demonstrate efficacy. So the harm of the non-challenge trial is either going to happen quickly (because we have crap mitigation) or slowly (because we have good mitigation) but it's going to happen, and the population itself may be suffering harm outside the study in parallel.
The only way around this I see that doesn't involve challenges is to relax the requirement for a double-blind demonstration of efficacy vs placebo by observing that the placebo group becomes ill.
I think I'm not clear on whether your argument is "you can't really have informed consent" or "this is deontologically immoral" (e.g. it's never okay to go infect people on purpose, regardless of how much harm you think you're fixing) or "there are too many unknowns to safely make this harmful action on utilitarian grounds?"
Yes, the studies clearly went faster because of the exploding infection rates. I guess the counterfactual is, in a world where we were mitigating properly, how quickly would it have taken to get a readout? The estimates were 3-4 months longer. So with much lower infection rates, we still would've had approval in early spring 2021, with production already having been ramped up to the point where most of the population could receive their first dose right away. That would be a much better scenario than our current one!
To respond to your question, my feeling is "there are too many unknowns to safely make this harmful action on utilitarian grounds" as you say. I think challenge trials have a place, but not with COVID. Proponents need to argue more clearly how it would've worked because I don't see how lives would've been saved (either in our current world or a world where the mitigation efforts had succeeded more broadly).
Thanks - that's clarifying. Would you say "challenge trials have a place but not with COVID _ever_" or "challenge trials have a place but not with COVID if it were managed non-stupidly?"
In other words (and this is a very, very hypothetical question because if I'm giving you a magic do-over card for how you think things should have gone, why can't you snap your fingers and go "competent mitigation") if it were a sadly fixed and foregone conclusion that we were going to have rampant community spread and hundreds of thousands of casualties, would that (own-goal pandemic) situation be fare game for challenge trials? Or still too much unknown?
I guess I go back to two key points here: (a) the long-term effects of COVID are potentially dangerous enough that we should be very, very careful about exposing people, and (b) I have yet to see a challenge trial design that gets us to fewer deaths/infections than our current state.
Essentially, you're saying "we would have willingly exposed X number of people to COVID in a challenge trial to save Y lives" knowing that 400k US deaths would have happened by now. In such a hypothetical universe, yes you can make that argument. But the particulars matter here. It's an amazing scientific feat that we got 95% efficacious vaccines into people 10 months after COVID was sequenced. Would it have really happened faster a different way?
Were the estimates really only 3 to 4 months longer to gather the data? If everywhere was Australia or New Zealand would it not take years? I think the key point of this entire discussion is that it took only a matter of weeks to develop a vaccine candidate (how smart are those people!) and the rest was just testing. We should be considering every possible approach to speed up testing and having debates about the ethics and science - which this message board is doing. Almost by definition the ethics will never be 'settled'.
Of course with a more definite and shorter timeline there would have been more political will for stronger NPIs.
But even with the same level of NPIs, getting a vaccine a day sooner would've saved thousands of lives. That should have huge moral weight. The ethical issues involved in letting *willing volunteers* get infected with covid are much smaller; maybe dozens of lives rather than thousands.
It's true there were (and are) a lot of unknowns about the effects of covid, but those same unknowns also weigh on the other side of the scale; by delaying a vaccine, you subject thousands of people to those unknown effects as the pandemic plays out naturally.
The questions about the design of the study are real, but it's hard to believe they couldn't have been figured out in a couple days. But we never got past the ethics issues to see.
Getting a *vaccine* a day sooner would have saved thousands of people. But getting *knowledge* of vaccine effectiveness a day sooner wouldn't have saved thousands of people, because vaccine distribution has been hampered more by manufacturing and logistics than by knowledge of effectiveness.
We didn't wait for knowledge of effectiveness to start manufacturing candidates, and we didn't have to wait for knowledge of effectiveness to start hiring injection workers and designing signup websites. It seems likely that instead of 14 million doses manufactured and 5 million given by the end of December, we might have had 2 million manufactured and 1 million given by the end of November, and 15 million manufactured and 8 million given by the end of December. But if we actually continue to ramp up injection capacity, then likely in a few weeks we will reach the same manufacturing bottleneck that we just would have reached sooner.
This is a good point.
I'm not sure these issues are as disconnected as you'd hope. For instance, the vaccine rollout was poor even though people had months to prepare. Prep time isn't as good as the real thing. I imagine it's psychologically (and probably legally?) easier to work on mass producing something which is already approved vs. speculative. But I'm not sure how big this effect is. I would love to read a breakdown of the manufacturing timelines and whether they were as fast as possible.
There is also some benefit to actually getting the doses out sooner (reduces more future infections).
This makes total sense to me, but then I've never seen the trolley problem as a difficult problem. You throw the switch, save net lives, sleep like a baby that night. There seems to be this human idea that sins we cause through inaction to be less bad than sins we cause through inaction, which I will never understand.
FWIW, what ethicists traditionally consider to be the "trolley problem" is not "what do you do when you see a switch that can kill one to save five?", but rather "why is it perfectly good to push a switch to kill one to save five, when it would be awful to shove one fat person physically onto the tracks to stop the trolley and save five?" There are many versions of the "kill one to save five" scenario that are discussed in the literature, and most people share the intuitions that in some of them you absolutely should do it, while in others you shouldn't.
(My take is that the examples are badly designed, and people are reacting to the fact that in the real world, the scenarios that people react badly to are ones in which you would be very far from certain that killing one would actually succeed in saving five - of course you shouldn't kill one for a small chance of saving five!)
That's a good point, a key difference between the trolley problem and real life is in the hypothetical scenario you have perfect information and there is no random chance involved.
Under what conditions are challenge trials going to be allowed? In policy, a clear bright line like "under no circumstances should you make volunteers intentionally sick" is valuable. This is effectively Matt's argument with vaccinating by age and the same argument applies with study design.
But the counterfactual world isn't a world where those people DON'T get the virus. In fact, it very deliberately is a world where they DO get the virus, since that's the only way to get the vaccine approved!
So... the argument is we should intentionally infect people in a controlled environment since people are getting it anyway?
I mean, yes, the vaccine readouts came much faster because the infection levels were so high. But that's not really a good argument to do challenge trials! If anything, it's an argument against, since you're not going to be able to ensure that people in such a challenge trial wouldn't be exposed outside of the trial guidelines.
Also, as others have pointed out, you don't get proper safety readouts without a placebo arm. And as we've seen, there are expected side effects. You can't analyze these without a control group for comparison.
Almost. I think the argument is that we should intentionally infect a _small_ number of people because a _large_ number of people are going to get it anyway. If non-challenge trial takes longer, the ethical cost of the non-challenge trial is what goes on in the rest of the community during the long trial if it turns out that the vaccine works (which has a > 0 expected value).
Not exactly... the cost is how long it takes to get people vaccinated. Which as we've seen, depends on manufacturing and distribution (not trivial considerations). Would any company have been able to produce more vaccines right now if they had had their readout 2-3 months earlier? Doesn't seem like that was the rate determining step, seeing as how they all ramped up production well before approval.
Right - you can argue against a consequentialist case for challenge trials on pragmatic grounds, but that argument can actually be settled in a way the philosophical one cannot.
Yes, they almost certainly would have been able to ramp up manufacturing earlier if they could expect approval earlier.
Evidence? The companies ramped up production at risk pretty early on in this case.
My understanding (from when I volunteered for the challenge trial) is that participants are kept in isolation, ensuring that they won’t expose others and also giving quick access to medical attention.
Not much for utilitarianism, huh?
Derek Lowe summarizes his argument as:
> I think that such trials are (1) still ethically questionable, (2) fraught with legal complications, (3) are for those reasons and others not likely to run meaningfully faster than regular Phase II trials, and (4) will produce data that may not be able to be extrapolated to the real-world situation.
Putting aside (1) as a question that will depend on your personal ethics, (2) and (3) are both about COVID-19. Matt is proposing to invest in solving those problems in advance of the
*next viral pandemic*. That proposal is entirely consistent with Derek Lowe's points.
(4) is a genuine factual disagreement. Derek Lowe presented an argument, and Matt presented a counter-argument. If you (@J. Willard Gibbs or others) don't think that counter-argument is convincing, why not?
A fair question and thanks for laying it out this way. My feelings on (1) seem to be counter to a lot of the folks here (there's a larger conversation on "right to try" drugs that needs to be had). I think much of the argument in this thread surrounds (2)/(3) but (4) to me is really the crux of it. So I'll state here as I have in other responses -- challenge trials have a place, but they are very, very situation-specific and COVID doesn't fall into that paradigm.
I guess surrounding (2)/(3) -- could we have actually developed something faster than we did using a challenge trial? Because we got these vaccines incredibly fast, to the point where manufacturing is going to be the holdup in getting people dosed and not the trials themselves.
But (4) is the big problem you have to solve. How, exactly, do you design a challenge trial that gets enough doses of a safe, efficacious vaccine into people more quickly than a traditional trial? Because every disease is a little bit different. You can use learnings from a previous vaccine (and with mRNA, we should!) but there's no guarantee that it will be safe or efficacious until you test it.
Traditional phase 3 (non-challenge) trials run the same set of risks, you just don't know what the confounding variables are. What was the weather like during the "real world" trial? What were local infection rates at the time? What was the market price of grapefruit? Do any of these matter? Are either of them divergent from the "real world" conditions that the vaccine will be deployed to the general public in?
The point of a traditional phase 3 non-challenge trial is that by including 30,000 people from a bunch of different sites in different nations in different hemispheres, you can create a study population that approximates the world average in a very large number of characteristics. Importantly, it approximates the world average in terms of the type and duration of exposure - which challenge trials very much would *not* (unless we've already done other trials to determine what sorts of exposure really have been the relevant ones in the world, which is something we absolutely should do, but would likely take longer than a year).
Exposure for the actual phase 3 trials happened under conditions where masks were mandated, many businesses closed, etc etc. It's not the same problem to the same extent, for sure. But I just don't see why the evidential bar has to be so high, for the FDA to just stop banning a medication and let people take it if they want. I think the "right to try" issues are really relevant to the question of the burden of proof here.
I apologize. No pissing match intended. I will try to back off of any rancorous tone I had.
I will not attempt to devise a specific study design. I am not a field expert. But I was under the impression that Lowe opposed Challenge Trials not due to implementation problems, but because he ethically feared exposing participants to a virus with unknown consequences. I’m sure there is an equally interesting and complicated debate to be had about how to properly implement a Challenge Trial but I am not doing so here.
I used the term “In the Wild” to somewhat glibly refer to the Non-Challenge Trial process. My understanding is as such:
- vaccinate some volunteers, call them Group V
- placebo some other volunteers, call them Group P
- let them all run around in the wild for a while
- once some portion of Group P have contracted the virus, this indicates there’s probably been sufficient exposure to Group V
- then check Group V to see if they fared better. If they did? You got yourself a vaccine.
In this sense, in a Non-Challenge trial, placebo participants are still getting exposed to the virus.
The only difference is it is passive exposure rather than active exposure. In a Challenge Trial you _actively_ expose them in a lab, in a Non Challenge Trial you _passively_ expose them on the streets of cities and towns across the world.
This is the first important point: you’re not “saving” any placebo participants from contracting the virus. For any statistically meaningful result (vaccine good? or bad?) some number of people in Group P is getting the virus.
The second important detail is that in the Non Challenge Trial, you require several non-participants, call it Group N, to also get the virus. These are not volunteers. These are ordinary people on the street. It is only through their exposure and transmission that you ever get any results in Group P or Group V!
For any Non-Challenge vaccine to come about, you must involve this non-volunteer Group N! Worse, Group N has to be _massive_ for there to be any statistically meaningful contact with Groups P and V.
When put in these terms, Non-Challenge Trials sound downright vicious! Not only must they involve non-volunteers, they involve _tons_ of them!
Thus, to many of people in this comment thread, they think who but a monster could oppose a Challenge Trial?! (again, assuming you could figure out implementation)
But indeed I don’t think you’re a monster. Nor Lowe. Here is what I think the core of this debate centers around and why many people seem to be talking past each other (and why I think this is such an interesting subject). We have different ethical frameworks and we think differently about active vs passive harm.
It sounds like to some people, the thought of actively harming Group P is worse than the thought of passively allowing harm to come to Group N. At the end of the day, I think this entire debate is that simple.
Personally, I would rather actively harm Group P (assuming they were volunteers) than passively allow harm to come to (a much larger) Group N of non-volunteers. That’s why I like the idea of Challenge Trials.
Okay, I'll bite:
Let's say (roughly) there were 30,000 people involved in a traditional trial. So 15k in the placebo group, 15k in the vaccine group (this is roughly the breakdown for Moderna/Pfizer/J&J/AZ). To achieve statistical power, the study designers determined that they would check endpoints when 30, then 60, then 90, then 120 people were infected across BOTH groups (again, roughly). Then they would compare the figures of people in who would be infected and measure the efficacy based on the ratio of people infected. So for example, if 80 people were infected in placebo group and 8 in the vaccine group, that's 90% efficacy (again, these were roughly the numbers you saw in Pfizer/Moderna).
Now if you ask me whether you're putting those 80 people at risk -- yes, you are! But now let's say you run a challenge trial and determine you need 10,000 people to achieve statistical power (I'm completely pulling that figure out of thin air... I don't know the actual number you'd need). So you expose 10,000 people to the disease, 1,000 get infected -- 90% efficacy. You've proven the same thing but exposed many more people to the disease. Obviously the numbers are arbitrary here... but are we really going to be able to get a safety readout amongst a cross-section of society with 100 people? Probably not. What about 1,000 people? Unlikely, but let's say for the sake of argument that you can -- that's still 100 people infected and you're greater than your placebo group in the non-challenge trial!
I just think that thinking about this as a trolley problem is getting a little too cute. You need a real world trial design that addresses these issues. It's not enough to say you're harming people passively versus actively -- I just don't see how a challenge trial reduces the harm at all.
I don’t see how you ever can expose fewer people to the virus in non Challenge scenario.
Suppose, as you say, you need ~120 placebo people to have contracted the virus before you’ve got yourself a meaningful sample. At this point you assume ~120 vaccine people also were exposed and so you’re saying you need an exposure group of ~240 people to make any meaningful determination.
You have two ways of exposing these people:
- expose each in an isolated lab (Challenge Trial)
- tell each to walk around NY until you’ve reached that 120 number
Practically by definition, the latter one is going to require that more than 240 people were exposed! By the time you’ve hit your 120 mark, thousands of New Yorkers would needed to have it in order for it to jump to those 240 people.
The numbers factor out. Call 120 N. For any N (short of some weird edge case where N is the size of the planet’s population) you’ll always limit exposure by doing the exposures in a lab rather than doing the exposures in the wild and waiting and hoping.
Except that you're only considering efficacy, not safety. You couldn't get enough of a safety readout on a vaccine with only 240 people getting it.
Then pick a higher N.
If you pick an N of 10,000 you will still require less global exposure by exposing each individually than if you sit around and wait for them to be naturally exposed by the rest of the world. Again- because for them to be naturally exposed millions of non-participants will need to get it.
You raise an interesting point about safety vs efficacy that I hadn’t considered. I suppose the goal there is to vaccinate enough people that you understand the dangerous side effects are under some low bar.
I don’t think this presents any categorically new problem in a Challenge vs a Non Challenge:
you can still choose to vaccinate as many as you want/need and you pick some lower subset of the group to expose.
The vaccinated but non-exposed people give you a safety readout. The vaccinated but exposed people give you an efficacy readout. You cross both with a placebo group so you have a baseline for both.
But does any of that change the fact that you’re still at the end of the day being more _efficient_ about exposure (thereby exposing fewer total people)?
Well now you have two arms to your trial, a safety arm and an efficacy arm. And you've complicated matters immensely. Is this going to provide a cleaner statistical readout than either a straight challenge trial or a traditional trial? And is it going to get you to your answer faster?
Again, Pfizer and Moderna both managed to produce safe vaccines that were 95% efficacious in 10 months, which beat the previous record for a vaccine by over three years. They were so fast that they couldn't even ramp up manufacturing to have enough doses ready by the time of approval (pharma thinks about this in great detail, and Pfizer's CEO Albert Bourla was very clear in the summer that they had already started manufacturing at risk). The onus is really, really high for designing a challenge trial that gets you where you need to be faster.
Right... and now more people have it.
yeah I've heard the argument that essentially boils down to "well COVID may be especially bad so we shouldn't do it" but like, COVID being potentially really bad is a great argument for why we should do it
Just a quick thought (PhD epidemiologist who studies ID): covid-19 is rather an unusually attractive candidate for challenge trials. You can find me musing about the use of challenge trials if you look closely enough. But it is a disease family for which vaccines had been developed (in animals but that's not nothing), for which there was a lot of research on vaccine targets because of SARS-1, and which had a relatively low fatality rate among young people.
Replace it with either smallpox or HIV and the arguments get a lot trickier.
Or Ebola!
From a cost benefit standpoint you also have to consider the manufacturing time to produce vaccines at a mass level. As far as I can tell, the longer period for a traditional phase three trial did not delay the manufacturing of either of the available vaccines. It's clear that approving either in August would not have delivered one hundred million shots in arms by the end of the year.
There is a whole separate problem of whether Operation Warp Speed was actually aggressive enough in ramping up manufacturing capacity. And the clear failure to prepare a robust enough delivery system.
Speaking as a fellow philosophy major (and one who's generally skeptical of a lot of what passes for bioethics), I'm not sure this article really engages with the ethical worry posed by challenge trials.
To be scientifically rigorous, as you acknowledge, a challenge trial would have to deliberately expose vulnerable, high-risk individuals to a virus with a substantial chance of killing them. That's different in kind from simply failing to use all means necessary to keep such people from getting infected (we never do everything possible to keep people from being infected, sometimes because of ethical constraints and sometimes because the measures would be too costly). Or to make the same point another way, people in the placebo arm of a normal vaccine trial are no worse off than non-participants in the trial; people in the placebo arm of a challenge vaccine trial will--reasonably often!--end up far worse off than had they ignored the invitation and chosen not to volunteer.
The response to this is, well, they consent to self-sacrifice. But we get a lot more dubious about consent when it comes to serious self-harm (see: hard drugs, dueling). It's really not like the medical personnel case, where there is rightly a huge amount of investment in trying to *keep* such personnel from being exposed to the virus--exactly what you can't do in a challenge trial.
There was a lot of effort to keep medical personnel from being infected, but when the PPE shortage happened, we didn't tell them to stay home. We accepted that they took a very substantial risk of infection in exchange for the benefit we would get from it. The social benefit of a challenge trial participant are way larger than the benefit of a nurse showing up to work in a cloth mask.
There are plenty of other better takes than mine on why we allow (and celebrate) people taking all kinds of risks. Medical professionals, military, law enforcement, astronauts, sports people, etc.
For fun though I'd like to take on the harder case... "we get a lot more dubious about consent when it comes to serious self-harm (see: hard drugs, dueling)"
Should we though? The (welcome) trend towards drug decriminalisation is explicitly recognising that people can take decisions which may be harmful. That's currently confined to 'soft' drugs but personally I'm in favour of it being extended to hard drugs.
In a similar vein look at combat sports (boxing, mma, etc), there's no question that they involve a high likelihood of serious harm and even death. Yet in large parts of the developed world we permit consenting adults to compete in combat sports. And again I personally see no ethical reason that couldn't be extended to dueling.
If you recognise an individuals sovereignty over their own body then the only real room for ethical objection is to say that people are incapable of making a valid decision because they:
1. have a mental disability/illness
2. lack information
3. are subject to some kind of coercion (actual coercion or much looser concepts like poverty)
4. are in some kind of catch-22 (only sane people are allowed to consent to being killed, only an insane person would consent to being killed).
There’s no social benefit to hard drugs or dueling. If you’re going to chide people for not considering the ethical issues you should be careful not to stack the deck by disregarding the benefits.
But that's not true, in our actual reality lots of people (likely most Americans) will never get COVID.
I mean, I guess you could try to select for the people who would seem likely to get it anyway, but if you're really good at that you could use that approach to reduce the time of the normal Phase III too. Might be pretty hard to get people >65 that way.
You actually can't do that! That proposal makes no sense at all! Transmission in nursing homes is a massive driver of deaths, it's vitally important to know the vaccine works in elderly people. (My recollection is that Trump had to threaten to fire the FDA head to get an EUA shortly before it would have issued anyway in a context where the logistics of vaccine distribution to states, let alone into people's arms, were not meaningfully sped up at all--appearances over substance as always with them.)
The problem with this is that for every member of the placebo group who gets COVID in a non-challenge setting, there are thousands in the placebo group who do not.
In the challenge placebo group, there could very well be a 100% chance of infection.
I'm still utilitarian enough to be vaguely in favor of challenge trials, but I don't think this particular argument really holds up.
Fair enough. I'll concede the point.
I don't know the right answer to the main question here (sc. "Challenge Trials: yea or nay?"), but I have a comment about the dynamics of the debate.
I suspect that people react to the choice between challenge trials and standard Phase 3 trials as though they are facing a choice between killing and letting die.
Some people think that distinction (i.e. causing harm vs letting harms occur) is ethically irrelevant, and indeed a source of ethical confusion. (This group is not perfectly coextensive with consequentialists, but fairly close).
Other people think that it is a distinction of fundamental ethical importance, which changes actions from permissible to impermissible: there are some things that one may not do, but may allow to happen. (This group is not perfectly coextensive with deontologists, but fairly close).
I believe the research on popular belief among non-philosophers shows quite a lot of support, more than majority, for the ethical importance of the killing/letting die distinction, and the causing/allowing distinction in general.
So you start your campaign of persuasion with some hard work ahead of you. It runs counter, or at least seems to run counter, to widespread and deeply held ethical intuitions.
It's not just killing vs letting die. It's killing with consent vs letting die without consent.
Other framing effects:
I bet poll results would differ a lot between:
A) should pharmaceutical companies be allowed to intentionally expose volunteers to the virus in order to speed up trials?
vs
B) Should volunteers be allowed to intentionally expose themselves to the virus in order to speed up trials?
It's "taking a risk of dying with consent." Do people who oppose challenge trials on ethical grounds also object to people donating a kidney or half their liver? If not, why not? I don't know the precise odds of dying or having a nasty outcome from that kind of organ donation, but I doubt it's on a different order of magnitude from the odds of a young person dying from covid.
Agreed, Alexandre. Those are factors to stress in the campaign of persuasion. They help to shift the frame of discussion away from one way of looking at it, towards others.
(MY already does this in emphasizing voluntary heroism).
And popular ethical reactions are often highly sensitive to framing effects.
This is a sort of inchoate comment on my part, so I apologize if it's hard to follow my reasoning.
I feel like there *is* a difference between killing and letting die, and that on some level consequentialism is not moral. But simultaneously that the government or public policy, dealing as it does with vast swathes of people, is inherently immoral in this manner. We do not live in a world in which societal policy (whether mediated through the government or some other institution) can be tailored to the unique situation of each person affected by that policy, and so... you've got to be consequentialist about public policy in a way that you don't and maybe shouldn't about some kind of individual action. The trolley of the thought experiment is different in kind to collective action.
Right - the consequentialist argument here is not necessarily "consequentialism is right", so much as "one side of the outcomes is so much more consequentially worse that consequentialism swaps other concerns."
If the number of people who get sick and/or die in the trials, in total, is the same, I think you'd have a nearly impossible time making the case for challenge trials. It's the harm to the rest of the community while you run the small trial that drives this discussion. And Matt's argument is that that harm is baked in, because if the larger community isn't having a dumpster fire like the US, the phase 3 will take longer and be harmful by duration.
I'm sympathetic, bsupnik, but "the consequences are so much worse that they swamp other ethical concerns" just *is* a claim that consequentialism is right.
Which is fine! That's the claim consequentialists should make! (And argue for, ideally.)
But you can't avoid it by saying, "I'm not saying cons. is right, I'm just saying those considerations override all other ethical concerns."
Ha ha ha ha I came so close to sneaking the conclusion into the premise. ;-)
I guess my thinking was more like "consequentialists think consequences matter and therefore may be able to make a relative moral ranking based on the measurable scope of consequences. So it's easier to argue against a consequentialist (after you accept their premise) if they are forced to admit their consequential moral differential is small."
You're right that this would be conceding the moral framework, but if you're not going to do that, you're not going to have an argument, you're going to talk past each other anyway.
No, I think there's an important difference between saying, "consequences matter," and "consequences are *all* that matters."
Yeah, that makes sense, and I bet a lot of people would agree.
Your rationale for it seems to blend together two different distinctions, though:
1) state vs private
2) large numbers vs small.
On the first rationale, states may employ consequentialist reasoning that private individuals may not -- on matters both large and small. The state may kill a thousand to save a million, or kill one to save five, even if that small-scale choice will never have larger consequences. Private persons may never do either, but states have a different moral status.
On the second, both states *and* individuals may kill a thousand to save a million, but nether states nor individuals may kill one to save five -- the numbers are just too all.
You could also have the view that both stipulations are necessary: no consequentialist reasoning unless you are 1) a state dealing with 2) very large numbers.
Which of these views do you incline to?
I think I have two different axes of thought here:
1. I am not sure that states, or other large organizations, should obey the same deontological rules that individuals do.
2. I also think that the error bars on consequentialist decisions are HUGE. And that outside of idealized thought experiments, even if you buy into consequentialism entirely, you have to deal with that. Which makes a "kill 1, save 5" thing problematic, but a "kill 1, save 10,000" thing much less so.
There are many questions about how public health officials evaluated costs and benefits to come to the decisions they mad. Why no challenge trials? is just one:
. Why no mass testing of asymptomatic people?
. Why no compensated self-isolation for asymptomatic people who test positive?
. Why the delay in approving cheap, fast screening tests?
. Why were recommendation of different kinds of social distancing, mask wearing, ventilation, traffic flow in different kinds of venues not tested to provide feedback on best/least cost practice?
. Why did officials not say for the beginning that recommendations would change as they learned more (not wearing/wearing masks, importance of hand washing/scrubbing surfaces)?
. Why the virtual silence on the point that social distancing is as much to protect others as to protect oneself. Why "Stay safe" instead of "Be safe"?
. Why was messaging not based on explicit, evolving models with recognition of how models were changing.
. If there are good answers to these questions, why have officials not supplied them?
The general impression is of total improvisation, as if the very idea of a pandemic was totally novel concept in public health.
.
Challenge trials are only more ethical in a very academic sense, which is why I think you're drawn to the topic. Obviously, killing people in an experiment vs people dying in an experiment are two very different things. And it's easier to talk about in the context of a low fatality disease like Covid, but what if the disease in question is overwhelmingly fatal? I guess you could get comfortable with killing people intentionally from a strictly utilitarian perspective, but I don't think that's where society's ethical code is at this point.
I think society doesn't support the strict mathematical utilitarianism that some people want to do. Kill one person to save two. Kill three to save four. You're going to have a tough time getting people to agree with those proposals, and, I think, correctly so.
But I do think that there comes a point where it becomes much harder to make the case against utilitarianism. Let's say we did challenge trials with 10,000 volunteers, and that resulted in moving up our entire vaccine timeline by 2 months.
That's plausibly something that kills less than 15 people (5,000 get the vaccine and of those maybe 125 get infected and due to milder infections none die, 5,000 get the placebo and 2,500 get infected, 12 die), and saves 150,000 lives. Each death saves 12,000 people. I think that kind of ratio washes away a lot of the ethical concerns we would have about killing one to save two.
I agree that the covid numbers make a pretty compelling case, but I still don’t see societal acceptance for intentional killing in the name of science. And when we look at more dangerous diseases it strikes me as a real non starter.
When you say killing people, the image that comes to mind is a scientist controlling a drone, injecting random people without their knowledge and seeing what happens. Overly dramatic, I know, but the point is you are overlooking the critical point of *they would be volunteers.* People do choose, regularly and of their own volition, to do something for the greater good even at great personal risk. People do this all the time. There are probably a dozen professions that in total make up a large percentage of the work force that would constitute a risky profession that isn't entirely justified by the pay. But obviously people do them anyway because they *want to help.*
Matt's comment towards the end about medical ethicists taking a dim view of self-sacrifice is spot on. It is a serious weakness in thinking. People like to do great things, and ethicists should let them - especially when the alternative is potentially tens of thousands of deaths.
Again, covid makes this an easier conversation. But let's say it's ebola, which is 90% fatal and it's a horrible death. I really don't believe there would be a long line of right thinking volunteers no matter what the math is, ie how many people my death would save. The people who would volunteer would overwhelmingly be suicidal and therefore not deemed capable of making this decision. Or you could open it up to people "volunteering" for large sums of money, release from prison etc but that is a whole other ethical can of worms. Probably the cleanest group to deal with is people with already fatal diseases with only a few months to live, but they would present obvious issues with the results. These professions you talk about are things like police I guess, where their risk of death is higher. But they also enter into the profession assuming they won't be killed, due to good luck or higher skill levels. And they are almost certainly right, because the probability of death is tiny. People entering an ebola vaccine trial will be virtually certain of death. I just don't see it as a valid comparison.
People might surprise you. Firefighting is majority volunteer position, for example. Extended periods of boredom punctuated by running in to a burning buildings sounds on paper like a really terrible idea, but here we are.
But in any case, I think something like ebola (with a very likely chance of death) might just a logistical exception and not a proof of a failure of ethics. Covid isn't ebola, and there would've been no lack of volunteers.
Right—why should the fact that even more dangerous diseases exist preclude the idea of doing challenge trials in this instance?
Well, if you can't get enough volunteers for a challenge trial, then, okay. No challenge trial.
But if we're in a counterfactual world where there's some horrifying ebola epidemic/pandemic, such that people are seeing their friends drop like flies to ebola, I think you might find a surprising number of people would be willing to do it. I might be wrong! Hopefully we never find out!
The scene playing in my mind is Armageddon, where the group of heroes know they stand a good chance of dying to save the planet, and they get "1 free wish + never ever have to pay taxes again" as the reward for their heroism.
How is a deadly pandemic not different than an asteroid on a collision course with Earth?
Good thing to ponder, especially given that at the start of COVID, with some of the scarier long haul stuff happening, we really didn't know if this would be much worse.
It seems like there would be more societal acceptance if experts were saying “actually this could save a lot of lives but it does raise certain ethical questions” rather than “This is a really bad awful idea with no merit whatsoever.”
…
I’m sympathetic to your argument here, but a lot of the anti-challenge arguments are very weak because they refuse to make the concession you do that “yes, it could very well save lives” that quickly makes folks like Matt and myself frustrated because it appears the medical community is being dismissive for condescending credentialist reasons.
Also slippery slope reasons—like comparisons to Tuskegee and Dr. Mengele get brought in a lot, even though they’re not at all apposite to a voluntary trial.
There's an outstanding question of whether we can be confident that people can meaningfully give informed consent for these trials, which does have some Tuskegee-like properties.
That's one element of this that I don't think consequentialist philosophy has much to say about.
How on earth is it "Tuskegee-like"? The Tuskegee subjects were simply lied to; they were told they were getting health care when instead they were injected with a disease. Whether or not it's possible to give informed consent for a virus trial—and I'm not sure why it's not—it's not within a hundred miles of giving people a disease without their knowledge or consent.
Having people engage in a risky behaviour, where the risks and population involved are high enough that deaths are probable is something we regularly do in the name of science.
https://en.wikipedia.org/wiki/David_A._Johnston
https://en.wikipedia.org/wiki/Space_Shuttle_Columbia_disaster
https://expeditions.fieldmuseum.org/antarctic-dinosaurs/antarctic-fossils
People are much more selfish then you give them credit for. Alot of people will happily let the volunteers die so that they and their loved ones don't suffer. After its over they'll mouth platitudes about how brave the sacrificed were and go back to voting against any tax increases to improve healthcare for poor people.
Was thinking... we knew the Apollo missions had a substantial chance of death, yet we did those in the name of science. I wonder how this is materially different, other than perhaps the scale.
Frankly I think a more deadly disease may make the idea even more popular. Take the same numbers as Michael uses with a IFR of 10%. That same trial would kill a lot more people (something like 260 people). But of course if the same number of people in the general population were infected with that fatality rate you'd have a lot more deaths, and the lives saved per death in the trial would be roughly equal.
Of course rough utilitarian calculations like that do not make for popular ideas, but with a virus that deadly, societies views on sacrifice would be WILDLY different. I think telling people we want to put some of you at great personal risk (something like 5% chance of death) for the sake of saving us all, in the context of millions of people already dying, is not an idea people with fear in the desperation and loss of life around them. Getting volunteers may be hard, but selling it to the public seems quite possible.
Killing *volunteers*. That part is important, and it's why this isn't just a utilitarian thing.
I'm a microbiologist by trade, and I think that it was a huge mistake to not to human challenge trials. Among other things, it's much easier to determine whether the vaccine induces sterilizing immunity, ie if people are protected from getting infected. And if they are infected, do they have infectious virus? It's also an efficient way of identifying which vaccine candidates are likely to be more or less effective. Still, human challenge trials are not enough and we'd still need phase 3 trials to detect rare events. The high fatality rate for older people makes human challenge studies that include people over 60 a harder question. (Then again, if a vaccine is >95% effective for people 18-55, it's almost certainly going to be pretty effective for people 65+.) But there are some challenges for human challenge trials:
We need to optimize infection: what is the inoculation dose, in other words, how many "live" viruses do we use? How do we infect them? Aerosols, droplets? Do we expose them to a high dose over a short period of time or a lower dose over a longer period of time? The world of immunology is full of infection models that are very reliable, ie the animal is reliably infected, but the infection method is somewhat different from how natural infection occurs, so it is only somewhat applicable to real-world infection.
This means that we'd have to spend some time optimizing the protocols before testing any vaccines. It's going to take quite a few volunteers.
If you run into trouble optimizing human challenge trials, phase 3 trials may end up being faster. But with a pandemic like this, you need to throw the kitchen sink at it.
I do think it's important to make sure that the volunteers are not pressured to take part and truly understand what they're doing. It's also important to remember that some subjects might die, and so you have to prepare the public for it.
I agree that we should do challenge trials, but this seems to almost entirely gloss over the actual hurdles to doing them:
1. Regulatory uncertainty. The FDA website says that companies need to discuss human challenge trials with the FDA before they are conducted. If that process is slow, then any potential time savings are wasted.
2. Public reaction. I think that the challenge trials are ethical, but how will the public react if someone died in a Pfizer challenge trial? Does Pfizer want to risk their reputation on that?
3. They are scientifically inferior because they don't test under real world conditions.
But yeah, my sense is people decided it was a bad trade-off (which you can disagree with) but then used ethics as the convenient, easy-to-explain reason to give on Twitter.
I happen to think that the biggest relevant objection is the epistemological objection 3 that Sam mentions in the other comment. But if one doesn't think that's such a problem, then 1 can be addressed, by an FDA planning committee deciding *this year* what sorts of conditions and regulations for human challenge trials would be allowed in *future* pandemics. (Presumably it would depend a lot on the nature of the risks of the relevant infection - we're probably more happy doing this with infections that pose a major risk for a short period that a good fraction of people recover from than with infections like HIV in the 1980s that pose a risk for a long period that people never recover from.)
Another way of saying this, if your phase 3 design means you know people are going to get infected, just over time, because they took the placebo, isn't it just more of a "passive aggressive challenge trial"?
One challenges via natural infection in society at large, the other does so under supervised medical conditions. If the difference is challenge can be faster, especially for things with an IFR under, say, 2-3% where you are using what appears to be a pretty safe technology in mRNA... it seems unethical to *not* do a challenge trial.
Downthread one objection is "but need to re-create natural infection conditions".
I would submit that a good study happening in phase 1/2 that aggressively works to find this out would be a good way to:
a.) Allow for challenge trials that "challenges" in a way that simulates natural infection
b.) More quickly understand the virus attack vector so we don't end up, say, doing a panic run on hand sanitizer when it turns out it's not terribly effective at preventing this specific disease
(generally, washing your hands is good, so maybe we got some nice side effect habits out of this... but that's another thread)
I think that some of these issues could be addressed by "challenge trials" of real-world behavior. Find known infected and known uninfected people, and then have them go to a bar together, or grocery shop together, and tell them to change their hand-washing practices, and figure out which routes of infection are most significant. Then, when the vaccine is ready, you also have good models of real-world exposure conditions, so that you're not using an unrealistic challenge of injecting people with the virus.