Using the RECOVERY trial is a sleight of hand that gives the game away... you're talking about steroids where the safety effects are broadly known and you don't need to do large scale toxicity or off-target monitoring. That's a huge reason why clinical trials are lengthy and expensive! Any new therapy is (correctly) going to have to be closely scrutinized for these reasons (you can't measure long-term effects without a time component). It's also why challenge trials, while theoretically sensible in certain circumstances, don't make sense for the vast majority of new drug approvals.
Also, we're years away from AI being able to be effective in this realm, for the reason that human biology is really, really complicated. Which is yet another reason (unaddressed in this piece) why trials are expensive: we keep discovering new pathways and targets which need to be accounted for. Then you have all of the drug-drug interactions, which also can't be well modeled, and that's another time sink.
Sure, but I feel like you're ignoring the specific issues raised in the piece, which mostly revolve around bureaucratic inefficiencies. One of their main recommendations is fixing the search interface on a website.
Yes, there's a lot of "boring" stuff that doesn't get implemented that could:
"Another failure mode we have observed is one where a generally valid perception of regulatory harshness leads players to overshoot, even in cases where rules are actually favourable to disruption. This leads to an overall tendency to stick to the status quo.
Such an example relates to reluctance on the part of CROs to implement cost-cutting measures like Risk Based Monitoring (RBM). Traditional clinical trial monitoring relies heavily on in-person site visits and comprehensive data verification and they are estimated to consume around 30% of trial budgets. RBM offers a smarter alternative by focusing resources on high-risk areas that most impact trial quality and patient safety. Studies show RBM can cut monitoring costs by more than half while maintaining trial integrity.
Yet despite FDA endorsement in 2013 and proven cost benefits, RBM adoption has been tepid. By 2019, while 53% of trials incorporated some RBM elements, few embraced its core components – only 10% implemented centralized monitoring and 15% reduced their data verification requirements.
Contract Research Organizations (CROs) swiftly pivoted to remote monitoring practices when on-site visits became impossible. While remote monitoring is not the same as central monitoring (a component of RBM), this adaptability suggests that broader RBM adoption is feasible given the right incentives. Still, the shift is far from complete: only 43% of studies starting in 2021 implemented centralized monitoring, and 27% did both centralized monitoring and reduced SDV/SDR. In surveys of the CRO industry trying to get at the bottom of this issue, fear of regulatory risk is a very often cited reason for not implementing these measures, something that was confirmed to us in 1:1 conversations with various actors in the space.
And it's not just that CROs themselves refuse to innovate. Sponsors (e.g. big pharma) are reluctant to face any risks (e.g. FDA rejecting their trial design), given the high cost of potential failure, and prefer the trodden path regardless of extra costs. I talked to Meri Beckwith, who runs a start-up called Lindus Health, focused on running faster and more efficient clinical trials"
If I understand the distinction between RBM and standard monitoring, it involves fewer in person visits by patients as well as some changes in how data is captured and verified. It would be good to separate the data-management aspects with changes to how patients are followed since the latter changes seem much more consequential and could make companies averse to implement these changes. Looking at it from a pharma company’s point of view, it seems foolish to try to save money on trials if it meaningfully increases the risk of failure. These lean trials might be good for academic or government funded studies comparing therapies but for a novel therapy, cutting corners on patient monitoring could result in random adverse events that would make the trial pause while an investigation is run. That is, what if a patient suffers a serious adverse event that is ultimately unrelated to the therapy? It could give the drug a black-eye or allow a competitor compound to move faster, ultimately resulting in billions of lost revenue.
I did ignore this in my comment, and I largely agree that bureaucratic inefficiencies can be streamlined (again, not my area of expertise, but seems to be what the studies suggest). However, this is not where the primary drivers of cost bloat lie; monitoring for safety gets extremely difficult the more we know about human biology since there are more pathways we need to watch out for, as well as more drug-drug interactions as people take more drugs. Additionally, measuring efficacy on new pathways and new chemical matter can't be streamlined by AI, and this is largely where the Phase III expenditures come in to play.
Yeah. I’m not sure that it’s actually exponential growth, but specialty drugs targeting novel pathways (and the associated drug spend) sure have ballooned over the past 20 years. I think everyone is on a specialty drug these days.
Side note: can we PLEASE (as I've been requesting for years) have Derek Lowe do a guest piece? He actually knows something about drug discovery and development and doesn't hand-wave away the critical details. It won't align with Matt's priors, but you know, it would do some good to have a different viewpoint here.
> Derek Lowe is an American medicinal chemist, author, and blogger. He is best known for his blog, In the Pipeline, which focuses on drug discovery, pharmaceutical research, and the science and business of developing new medicines. Lowe has worked in the pharmaceutical industry for decades, specializing in areas like drug development and organic synthesis. His writing often provides insights into the challenges of pharmaceutical R&D, commentary on industry trends, and explanations of complex scientific topics in an accessible way. His blog is widely respected among scientists and industry professionals.
I may add my own biased justification should time permit. It'll be summarized as, "Lowe's is a brilliant medicinal chemist and his Pipeline blog distills the down a lot of complexity. The Ben Thompson of chemical medicine!"
Correct, although safety is still monitored throughout Phases II and III. As I stated in a different comment, there aren't good ways to measure efficacy in new biological pathways that don't take time.
Remember when as hundreds of Americans were dying each day the FDA was doing its own trial analysis of the drug that would save them? It's about accelerating that review and streamlining recruitment not changing the trials themselves.
Presumably you're talking about the COVID vaccines, in which case, remember the 100+ trials that didn't pan out? Would you have signed on to one randomly knowing that most of them were abject failures? Or would you have gladly taken Merck's (arguably the most reputable pharma company historically), which was also a failure?
Merk discontinued in a Phase I trial given a clear correlate namely Ab titers. The strong argument for delay would be the adenovirus vector vaccines that caused strokes, but the Phase III trials were not able to detect something that rare, and there's nothing the FDA could do beyond demand expanded ones.
I'm specifically talking about how the trials reached their readouts in August, but the approval didn't come until after Thanksgiving. There's no reason that has to be the case. The FDA was doing a ton of reanalysis rather than focusing on the most likely risks and thinking about how to cut parts or move them left of the unblinding.
I'd quibble with your timeline there... data readout is not instantaneous! But a fair point that it likely could've been moved up by a few weeks.
Merck's trial was interesting though... their CEO (Ken Frazier) had come out during the summer warning the world that the vaccines would likely not be ready until at least 2021 and likely beyond, based on their initial readout. Meanwhile Albert Bourla of Pfizer was saying the exact opposite, based on theirs. Obviously Pfizer won the race, but even at the time I was fascinated (and highly concerned) that Merck was so willing to make that statement based solely on their internal data.
I remember that review meeting being scheduled after Thanksgiving and thinking that the cost-benefit of doing it even a few days sooner would have been huge, holiday be damned.
Let's not forget the CDC advisory council had to meet after the FDA and there was a day in between because of their day jobs. The army should have been putting them on flights to DC that night.
FWIW, I signed up on One Day Sooner, and I would have done a challenge trial for probably any of the candidates. But COVID was a unique event*, and I don't think there's nearly as much utility for challenge trials in most cases.
I still don't think the case for challenge trials for COVID was as strong as its proponents made it out to be; remember, the first vaccines were approved over three years faster than the fastest ever to date, not to mention that even had they been approved mid-summer, the manufacturing delays wouldn't have allowed access any more quickly. But I don't dispute that even moving things up 1-2 weeks wherever possible would likely have saved lives, and it's probably something than someone has calculated.
Nobody cares how hard you try. We could have approved on animal models+human immunogenicity back in April just like we do for vaccines to some really nasty things.
Manufacturing delays were largely downstream of approval issues and unwillingness to pay the amount the vaccines were worth. And now we've dropped the capacity. We should be able to respond to a pandemic in 60 days.
“Manufacturing delays were largely downstream of approval issues…”
The Wall Street Journal had a really good long-form article back in December 2020, about the entire vaccine effort at Pfizer. They were producing doses of vaccine as fast as they could starting in the spring of 2020 when they first knew they had a good vaccine:
“Starting in March, Mr. McDermott spent $500 million to buy and design equipment, more than double what he budgeted. Pfizer retrofitted an Andover, Mass., plant to make the mRNA, and assigned a St. Louis factory to make the raw materials for the shots. Numerous times it returned to suppliers or contractors to increase orders.
“Instead of making mRNA in giant steel tanks, which would add months, Pfizer used disposable bags. Each fits at least 500,000 doses.
“In April, the company began buying machines the size of a single-car garage that play a crucial role in production.”
I don’t think we’re years away from AI being effective in this realm. The progress in even the last six months in using LLMs for genomics has been astonishing. The piece talks about something much further along, using LLMs for automatic redaction - that seems completely feasible today.
I think AI will *increase* costs rather than decrease costs. But for good reasons - AI will be better at helping us design new molecules for new targets but that will exacerbate costs by giving us more and better specialty drugs for new conditions.
You would be shocked at how expensive white-collar mistakes are at Pharma companies. Documentation inaccuracies in Clinical Trial Protocols mandate protocol re-approval, and can be the source of hundreds of thousands to millions of direct and indirect costs to trial sponsors.
+ This. There is a growing body of Academic/Industry research on clinical trials + AI (ie Jimeng Sun @ UIUC, James Zhou @ Stanford, Hoifung Poon @ Microsoft) exploring a myriad of topics.
A lot of the application research takes an expensive, technically challenging problem, and attempt to formalize it by creating a dataset one can benchmark against, and proposing a non-naïve solution using state of the art ML methods.
The problems approached are much broader than just drug discovery, but also operational challenges like patient to trial matching, publication evidence synthesis, statistical analysis.
Also, look at roles at top-20 pharma companies. Most of these organizations have their own technology and AI teams that are trying to cut operational costs using LLMs and traditional ML methods.
As someone who doesn't live in America, one thing I've always wondered about is why there's so much focus on what the American regulatory regime does.
Like, can't these global drug companies go to Vietnam or Nigeria or Indonesia (or even Japan) and work with the government there to get fast, cheap, effective trials in place with millions of people?
I know the answer is probably something like: the drug companies only care about selling to Americans for exorbitant amounts of money and aren't interested if they can't do that so that's why they don't do that but still..... There's this weird default assumption that America should pay all new drugs that the entire planet gets... Even among people that probably wouldn't agree with that if you asked them explicitly.
Pharma companies do run trials in the countries you listed. But they still need to meet the FDA's standard of evidence to get approval in the US (same thing in the EU). And one underrated factor here is that the US has a much lower level of corruption than other places... so the regulatory agencies will raise a skeptical eyebrow toward data that comes from a place where those norms are not as strong.
My understanding is that there's also the concern that if drug trials are done only in poorer countries with less oversight, less institutional experience with quality clinical trials, etc. that the data would be suspect (if not outright fake). There's also the PR problem for what happens if a rich country pharma company tests out a drug that turns out to be low quality with major side effects in a poorer country.
There have been some instances where drug trials in low or middle income countries used an inferior control arm— ethics requires you to use standard of care in the control arm, but the control arm was not standard of care. This can make the investigational drug look more effective than it is.
That’s not to say this can’t happen in rich counties, but it’s less likely to happen in rich countries with strong regulators
Most trials are global in nature and include sites across many countries. You wouldn't want to test a new drug in only 1 country because the results may not be applicable to different populations. The countries you listed can be part of a global trial, but can't be the totality.
The biggest issue with running trials in poorer countries is they don''t have the infrastructure to manage a clinical trial. You need staff to handle the extra paperwork, equipment for different tests, ability to store and ship biological samples etc. Also, in certain diseases it matters what standard of care patients have access to. For example, cancer drugs show proof of concept in later lines of therapy. A patient in Nigeria whose disease wasn't resistant to the standard of care drugs available in the US and EU wouldn't be helpful to determine the efficacy of a new drug. Patient retention can also be a bigger problem.
“ Like, can't these global drug companies go to Vietnam or Nigeria or Indonesia (or even Japan) and work with the government there to get fast, cheap, effective trials in place with millions of people?”
Pharma executives, eager to get their bonuses and cash in on their stock options, would conduct regulatory arbitrage and push fake trials in whatever country would look the other way.
I don’t think the aducanumab example supports the point you’re citing it for; more the opposite. The share price of Biogen suffered in the wake of the failure of that product and the CEO who championed it lost his job.
They came astonishingly close to robbing the entire US healthcare system blind with a drug that was known to not be effective. It was a very, very high expected value bet.
Yes, like BronxZooCobra originally said, there's a regulatory arbitrage. That usually implies a government failure of some kind. And the result is that in some not terribly uncommon cases "peddling ineffective and/or unsafe drugs would boost their share prices".
My brother is a 737 pilot for a major US carrier. His take is that the crashes were mostly a pilot-training issue, and there's a reason they happened in poorer countries.
For the specific MAX crashes, yes, that's true. But they're evidence that the larger culture at Boeing had changed from being run by competent airplane engineers to the financial executives. You can either focus on planes being cheap or being safe, and Boeing switched their priority to disastrous effect.
Nope - the final report of the second crash was the plane almost instantly entered an unrecoverable state. The aerodynamic forces on elevator were too high to manually trim.
American and French investigators both released comments stating that the flight crew did not follow correct emergency procedures. The Ethiopians apparently disagreed.
There are two parts to your question that are usually tied together, but are worth considering separately:
1. Why do we focus on how clinical trials are run in the US?
2. Why do we focus on how drugs are approved in the US?
There is no rule that says -- and in fact it is often not the case -- that drugs must have solely (or even predominantly) US trials to be approved in the US. International trials are under-done because of risk aversion among trial designers, as Ruxandra discusses a bit in her companion post (https://www.writingruxandrabio.com/p/on-clinical-trial-abundance). But, for the ones that are (or for the minority US component of a multinational trial), it's worth focusing on the US because it can be improved.
The second question -- can't we focus on getting drugs approved elsewhere? -- is basically at the mercy of the economic fact that the US does half the world's healthcare spending, and an even greater fraction of the world's spending on on-patent drugs (meaning, their first ~ten years on the market, before competition drives their cost down to close to the cost of production). Speaking as someone who does financial modeling for drug developers, the accepted wisdom is to make sure your drug is a good investment if you consider just the odds of US approval and the size of the US market, then consider the secondary markets (probably starting with Europe) as nice add-ons.
*Should* this be the accepted wisdom? Well, it is. The US is the richest country in the world (by any reasonable metric) and by a large margin, and so on some level it can afford to (in the same way that it can bankroll a substantial portion of work against AIDS in sub-Saharan Africa). On another level, I don't agree with the US-pays-everyone-freerides system, but it's not like I have any personal leverage to convincing Europe to pay more for their healthcare, so what's the use? We work with what we have.
Trials are commonly conducted in Eastern Europe for the reasons you state. Typically, costs per subject are much lower.
The challenge with other countries is that there are real, practical barriers to conducting good studies there. For example, it can be very hard to import all of the drugs, devices, and supplies needed. It’s common for a trial to use a device that isn’t approved in all countries and it may not be possible to get it through customs in a place like Vietnam. Another challenge is having consistent results of blood tests (labs). when different laboratories use different equipment, the results can’t always be compared on an apples-to-apples basis. Because of this, many drug trials use central labs where they ship all blood samples to a single lab. But that is also dependent on shipping the samples in roughly the same time. That’s very hard to do from somewhere like Nigeria. And, clinical study investigators aren’t made equal. In countries that don’t do many trials, enrollment or even the results could be impaired by having newbie investigators.
Given all this, there’s a minimum set of capabilities needed to do a lot of high-quality drug trials. there are not many countries with lower cost and these capabilities. None of this has to do with regulations.
They already go to other countries to do trials and studies...
Additionally, some countries in the EU and of course Japan are actually stricter in some ways related to drug development and clinical trial management than the US is. however there are trials happening in other countries, and in order to meet the FDA standards related to the clinical studies and new drug application they need to do certain things to sell in the US market.
I feel like I don’t understand the trade offs argued for here enough to know if this is persuasive. The whole article has a bit of a Chesterton fence problem.
I don’t understand the specifics of the rules we have well enough to say if I’m for the reforms listed here.
The tradeoffs are speed/efficacy in approving new drugs vs. higher risk of unforeseen adverse effects, including possibly extremely bad ones. Remember thalidomide?
People tend to have a status quo bias and generally judge sins of commission much more harshly than sins of omission. So, "this drug could have been approved but out of an abundance of caution it wasn't, so people with this illness will continue to suffer/die for lack of effective treatment" = it's just the way things are. "The FDA approved this drug and it unexpectedly caused fatal heart attacks in 0.1% of users" = this is an outrage, the FDA are criminals (and their director probably deserves to be shot, at least according to some edgelords on X).
To clarify, I am explaining, not excusing. It's likely that the status quo is TOO risk averse. Scott Alexander has made this point on his Substack, ACX.
There was a heartbreaking series of posts on ACX by a patient with advanced cancer (forget which type) who would write about the difficulties and hassles of enrolling in clinical trials. The patient has since died of cancer, leaving behind his wife and young child. He had become a bit of a celebrity in ACX world and there was an outpouring of sympathy at his death.
I understand that you're explaining, not excusing, but your comment doesn't really come across as "hey Ruxandra and Willy? here's some advice, you might not have realized it, but you're likely to run into resistance regarding..."
It's certainly true that risk aversion and omission/commission bias and medical history that was rewritten for political ends are headwinds -- but do you think they are reasons that Ruxandra and Willy should *give up on* the clinical trials abundance agenda? Or are you trying to point out specific things that they should be more aware of / cautious about than they currently are being?
This piece was too dry to be interesting and too jargony to teach me much. It might have been salvaged by making the proposals more than bullet points and focusing on the most important three or four.
Thanks for highlighting. There's more detail here (and at least they actually note that RECOVERY was on dexamethasone, which is a repurposed drug with a known safety profile), but it still fails to address the problem of discovering unknown safety issues with new chemical matter and/or new biological pathways. None of that is something that you can hand wave away with AI. You need to test the drugs for long term effects, and that simply takes time!
Maybe this was misinterpreted. We do not think all trials can be like RECOVERY, but RECOVERY itself would have taken much longer if the urgency spurred by COVID wouldn't have allowed them to bypass a lot of cumbersome administrative requirements, which is something Martin Landray talks about. So, obviously, a trial for a new drug with unknown safety profile won't take as little as RECOVERY, but that does not mean there are no efficiency gains to be made!
To what extent is this just a question of risk aversion? Would it be worth accepting a higher risk of long-term adverse effects to get more new drugs more quickly? Do we even have enough data to understand the trade offs?
The medical profession seems to default to standard of care rather than engaging tradeoffs. A young doctor friend of mine urged me, 47 years old, no family history of cancer before 80, to have a colonoscopy. I said it sounded really unpleasant and I don’t want to shell out deductibles to fast and have a camera shoved up my ass. I asked “what are the odds I have colon cancer.”. She had no idea. I consulted Dr. Google and pegged the odds that not getting s colonoscopy kills me at about 1 in 5000. I’ll take the risk! I really wish the medical profession were more willing to discuss trade offs and to tweak the care different patients get based upon their risk tolerance.
>the odds that not getting s colonoscopy kills me at about 1 in 5000. I’ll take the risk!
I think you haven't accounted for your potentially getting colon cancer (God forbid) but it's found later, without a colonoscopy, after symptoms like blood in your stool. You might still live, but after a very unpleasant year or two of radiation targeting your GI, chemotherapy, in the worst case major bowel surgery, etc.
If you have to have major surgery to address late-stage colon cancer, there's a nonzero chance that you have to live with a colostomy bag for the rest of your life. You know what that is? Want to consult Dr. Google on that?
The point of the colonoscopy is to find & treat it early. You can 'survive' later-stage cancer but at a terrible QOL. Not sure you've properly evaluated all of the facts here
Eh a friend got her first colonoscopy at 47 and they found a tumor the size of a grapefruit.
I’ll also add death isn’t the only bad thing prevented. They also remove polyps while they are in there which, while not cancer, can turn into cancer. Getting a polyp removed vs. having to have your colon removed and spending the rest of your life with a colostomy bag. Sure they successfully treated you with surgery, radiation and chemo and you didn’t die. But having colostomy bag for the rest of your life is still going to suck.
A colonoscopy with a twilight sedative, money and time aside (mine was public-funded), is as they once said “a nothingburger”. And they chop off the polyps before they potentially become a problem.
My husband had a routine colonoscopy recently. The day before the procedure he couldn't eat any solid foods and he had to drink a gallon of tasteless saline to flush out his colon. The doctor had told him frankly: "This will make you poop constantly all day, make sure you spend the whole day near a toilet." And so he did.
I 100% agreed that he needed a colonoscopy, and I'll have one too when my time comes (I'm 4 years younger than my husband), and a colonoscopy >>>>>> painful death of colon cancer. But let's not sugarcoat it. Getting a colonoscopy sucks ass.
I applaud you making a reasoned choice that fits with your priorities.
I wouldn't -- and haven't -- make the same choice. The procedure itself is a nothingburger. The day before is unpleasant to be sure but a pretty small price to pay for peace of mind, imo.
Therein lies our difference. I am resigned to the fact that it’s all going to fall apart. My vitality will wither. There will come a point where I hope I have the balls to blow my brains out as the alternative is worse. Good health is evanescent. Sacrificing an evening to fasting and an afternoon of recovery and a few grand in copays for “peace of mind” is poor value. My life is barely worth living, I don’t want to sacrifice happiness to extend it.
The implication is that the admin requirements are what lead to the long timelines and large expense. It is absolutely a factor (I agree with your points on this, though my knowledge on this is peripheral so I take your word for it). But the key driver of long timelines is (a) examining efficacy in new systems and (b) examining safety effects. Neither of these things can be done without time, and these get longer and more complicated the more we learn about human biology. AI as of now can't solve this -- it's not even capable of finding new biological targets (the majority of papers published around this are proof-of-concept studies around repurposed kinase targets).
Details and questions aside I am very happy to see this issue addressed. Given how fundamental good health is to well-being (selection bias in day to day life hides just how many people’s lives are degraded by poor health outcomes) I’ve often wondered why we don’t allocate significantly more public money to medical research but of course it’s all important to get the most bang for your buck too.
Great stuff! I like bringing in experts to discuss issues like this rather than just relying on us generalists (i.e., strong on opinions, not so much on facts and expertise).
I look forward to a well-led FDA and HHS vigorously pursuing these important reforms starting on Jan. 20, 2029.
One thing I'm curious about, for people who have experience doing clinical research: in this essay, Scott Alexander tries to do a simple study (comparing the results of an existing psychological screening questionnaire to a psychologist's expert judgment, in the process of seeing the psychologist's existing patients; no plausible mechanism by which participants can be harmed.) He describes a horrific amount of pointless obstacles raised by his Institutional Review Board, which explain why no one does simple and harmless studies. https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
My question is, is this at all representative? Are truly pointless bureaucratic obstacles a major impediment to clinical research? If so, that seems like low-hanging fruit to remove -- and as long as it persists, it needlessly hinders evidence-based medicine and denies patients access to more effective diagnostics and treatments.
One factor that distinguishes Scott's story from -- I'm sorry -- professional clinical research, is that he was basically limited to the procedures at [specific hospital], and is S.O.L. if they don't prioritize their office of clinical research (including IRB); a developer-sponsor would pick from across the country (ideally, the world, but often they stop at "the country" for a variety of good and not-good reasons) for centers whose research offices do...better than Scott's hospital's did.
It is not *easy* to move the football forward ten yards, but it is not quite as hard as you might think if your one attempt was to run straight into the middle of the defensive line. (The opposing team is the terrible brokenness of the world, not any coalition of people who are actually evil. Usually.)
This certainly looks reasonable to an outsider. I wonder if it woud not be worthwhile asking, why, if these things make senses, the relevant agencies have not already done them? I suspect it is that like other agencies they do not have Prime Directive that regulations must pass a cost benefit test.
Dropping down a level of specificity, I was surprised by the absence of international cross approval among agencies and reference to best regulatory practice in other countries.
It's because the FDA has an incredibly risk adverse internal culture that ignores Americans dying while they do their jobs. They literally delayed the biological insulin approval by months just for the PR issues if it did in fact cause problems. Also the kidney guy who approved no drugs and would sandbag on the beltway. After he retired thousands of American lives were saved by the drugs he hadn't acted on.
Phenomenal guest essay. Especially the additional policy memo's that go into more detail. We need the groups like PhRMA and the Biotechnology Innovation Organization to get on board and actually use their lobbying power for good stuff like this. There are lot of operational efficiency opportunities in these memo's.
On the other side, there seems to be a huge disinformation push from some figures in the media world that don't trust any clinical trial data, or efficacy information until they of course get sick or their family needs a drug. Additionally, I think that too much venture funding is in the emerging biotech space, and because a rejected trial may affect stock prices they don't necessarily want to take risks on multiple assets that aren't billion dollar drugs. It's all a very complicated process, either way making trials easier to administer and manage would go a long way.
I still don't think Generative AI will solve every problem, since you still need wet labs, dry labs and of course actual clinical data. We've solved a lot of the big problems, but novel therapies, oral treatments, toxicity levels and of course trials that look into long term adverse events are still needed in my opinion, however Generative AI will and is currently a gamechanger on the discovery and R&D side.
I wholeheartedly support RCTs in medicine and elsewhere. However, I think the article does not put sufficient focus on a fundamental RCT limitation: time-to-outcome. Example: "Some clinical trials are unavoidably expensive and difficult: for example, trials with new biologics have to be manufactured in small batches, administered with physician supervision to watch for unknown adverse events, and involve collection and analysis of many biological samples per patient." There is no mention that in many diseases (most cancers, for example) you have to wait many years to observe outcomes. The incremental innovations described here, while very welcome, do not change this fact. I think this ought to be a significant part of the discussion. At the very least, it should be pointed out that the proposals will not make a tangible impact in some important disease areas. I am pretty sure the authors are aware of this, but not everyone is, and it should be made clearer, in my opinion.
For context, I am an AI scientist working on clinical applications. I worked on cancer AI for 13 years before switching to infectious diseases literally because the cancer RCTs take too long. I worked on this trial that took 6 years to execute (and yielded negative result):
A big benefit here would be a national death registry and a law allowing pharma companies to access this data for any trial participants. List to follow-up and censuring are a challenge for cancer trials specifically.
Some things that stood out to me were: the cost of patient recruitment due to overly specific criteria and the FDA's insistence on American trials. Both things that got sidestepped in some of the successful COVID era trials.
Correction (as the podcast guest): I don't think that that FDA insists on American trials! I think that this is a misconception by over-cautious trial sponsors, as Ruxandra discusses in her companion post: https://www.writingruxandrabio.com/p/on-clinical-trial-abundance
I agree that Covid-era therapeutics trials, with one exception, fell into the buckets "under-recruited and failed" or "happened outside the US" (edit: or both), and that that experience is absolutely worth learning from.
Using the RECOVERY trial is a sleight of hand that gives the game away... you're talking about steroids where the safety effects are broadly known and you don't need to do large scale toxicity or off-target monitoring. That's a huge reason why clinical trials are lengthy and expensive! Any new therapy is (correctly) going to have to be closely scrutinized for these reasons (you can't measure long-term effects without a time component). It's also why challenge trials, while theoretically sensible in certain circumstances, don't make sense for the vast majority of new drug approvals.
Also, we're years away from AI being able to be effective in this realm, for the reason that human biology is really, really complicated. Which is yet another reason (unaddressed in this piece) why trials are expensive: we keep discovering new pathways and targets which need to be accounted for. Then you have all of the drug-drug interactions, which also can't be well modeled, and that's another time sink.
Sure, but I feel like you're ignoring the specific issues raised in the piece, which mostly revolve around bureaucratic inefficiencies. One of their main recommendations is fixing the search interface on a website.
Yes, there's a lot of "boring" stuff that doesn't get implemented that could:
"Another failure mode we have observed is one where a generally valid perception of regulatory harshness leads players to overshoot, even in cases where rules are actually favourable to disruption. This leads to an overall tendency to stick to the status quo.
Such an example relates to reluctance on the part of CROs to implement cost-cutting measures like Risk Based Monitoring (RBM). Traditional clinical trial monitoring relies heavily on in-person site visits and comprehensive data verification and they are estimated to consume around 30% of trial budgets. RBM offers a smarter alternative by focusing resources on high-risk areas that most impact trial quality and patient safety. Studies show RBM can cut monitoring costs by more than half while maintaining trial integrity.
Yet despite FDA endorsement in 2013 and proven cost benefits, RBM adoption has been tepid. By 2019, while 53% of trials incorporated some RBM elements, few embraced its core components – only 10% implemented centralized monitoring and 15% reduced their data verification requirements.
Contract Research Organizations (CROs) swiftly pivoted to remote monitoring practices when on-site visits became impossible. While remote monitoring is not the same as central monitoring (a component of RBM), this adaptability suggests that broader RBM adoption is feasible given the right incentives. Still, the shift is far from complete: only 43% of studies starting in 2021 implemented centralized monitoring, and 27% did both centralized monitoring and reduced SDV/SDR. In surveys of the CRO industry trying to get at the bottom of this issue, fear of regulatory risk is a very often cited reason for not implementing these measures, something that was confirmed to us in 1:1 conversations with various actors in the space.
And it's not just that CROs themselves refuse to innovate. Sponsors (e.g. big pharma) are reluctant to face any risks (e.g. FDA rejecting their trial design), given the high cost of potential failure, and prefer the trodden path regardless of extra costs. I talked to Meri Beckwith, who runs a start-up called Lindus Health, focused on running faster and more efficient clinical trials"
“…reluctant to face any risks (e.g. FDA rejecting their trial design)”
Are there any practical reasons (as opposed to regulatory reasons) that the FDA can’t pre-approve trial designs to limit such risks?
If I understand the distinction between RBM and standard monitoring, it involves fewer in person visits by patients as well as some changes in how data is captured and verified. It would be good to separate the data-management aspects with changes to how patients are followed since the latter changes seem much more consequential and could make companies averse to implement these changes. Looking at it from a pharma company’s point of view, it seems foolish to try to save money on trials if it meaningfully increases the risk of failure. These lean trials might be good for academic or government funded studies comparing therapies but for a novel therapy, cutting corners on patient monitoring could result in random adverse events that would make the trial pause while an investigation is run. That is, what if a patient suffers a serious adverse event that is ultimately unrelated to the therapy? It could give the drug a black-eye or allow a competitor compound to move faster, ultimately resulting in billions of lost revenue.
Thanks for this! Do you happen to have a reference link for these studies?
They’re all linked in my post
https://www.writingruxandrabio.com/p/on-clinical-trial-abundance
I did ignore this in my comment, and I largely agree that bureaucratic inefficiencies can be streamlined (again, not my area of expertise, but seems to be what the studies suggest). However, this is not where the primary drivers of cost bloat lie; monitoring for safety gets extremely difficult the more we know about human biology since there are more pathways we need to watch out for, as well as more drug-drug interactions as people take more drugs. Additionally, measuring efficacy on new pathways and new chemical matter can't be streamlined by AI, and this is largely where the Phase III expenditures come in to play.
Yeah. I’m not sure that it’s actually exponential growth, but specialty drugs targeting novel pathways (and the associated drug spend) sure have ballooned over the past 20 years. I think everyone is on a specialty drug these days.
Side note: can we PLEASE (as I've been requesting for years) have Derek Lowe do a guest piece? He actually knows something about drug discovery and development and doesn't hand-wave away the critical details. It won't align with Matt's priors, but you know, it would do some good to have a different viewpoint here.
Noted!
Seconding Derek Lowe. I'm feeling lazy so here's ChatGPT's unbiased description
> Derek Lowe is an American medicinal chemist, author, and blogger. He is best known for his blog, In the Pipeline, which focuses on drug discovery, pharmaceutical research, and the science and business of developing new medicines. Lowe has worked in the pharmaceutical industry for decades, specializing in areas like drug development and organic synthesis. His writing often provides insights into the challenges of pharmaceutical R&D, commentary on industry trends, and explanations of complex scientific topics in an accessible way. His blog is widely respected among scientists and industry professionals.
I may add my own biased justification should time permit. It'll be summarized as, "Lowe's is a brilliant medicinal chemist and his Pipeline blog distills the down a lot of complexity. The Ben Thompson of chemical medicine!"
My understanding is that safety trials (phase 1) are usually the smallest
It's usually efficacy trials, you're phase three trials that are the biggest
Correct, although safety is still monitored throughout Phases II and III. As I stated in a different comment, there aren't good ways to measure efficacy in new biological pathways that don't take time.
agreed. But we shouldn't be adding to that time with needless red tape.
Remember when as hundreds of Americans were dying each day the FDA was doing its own trial analysis of the drug that would save them? It's about accelerating that review and streamlining recruitment not changing the trials themselves.
Presumably you're talking about the COVID vaccines, in which case, remember the 100+ trials that didn't pan out? Would you have signed on to one randomly knowing that most of them were abject failures? Or would you have gladly taken Merck's (arguably the most reputable pharma company historically), which was also a failure?
Merk discontinued in a Phase I trial given a clear correlate namely Ab titers. The strong argument for delay would be the adenovirus vector vaccines that caused strokes, but the Phase III trials were not able to detect something that rare, and there's nothing the FDA could do beyond demand expanded ones.
I'm specifically talking about how the trials reached their readouts in August, but the approval didn't come until after Thanksgiving. There's no reason that has to be the case. The FDA was doing a ton of reanalysis rather than focusing on the most likely risks and thinking about how to cut parts or move them left of the unblinding.
I'd quibble with your timeline there... data readout is not instantaneous! But a fair point that it likely could've been moved up by a few weeks.
Merck's trial was interesting though... their CEO (Ken Frazier) had come out during the summer warning the world that the vaccines would likely not be ready until at least 2021 and likely beyond, based on their initial readout. Meanwhile Albert Bourla of Pfizer was saying the exact opposite, based on theirs. Obviously Pfizer won the race, but even at the time I was fascinated (and highly concerned) that Merck was so willing to make that statement based solely on their internal data.
I remember that review meeting being scheduled after Thanksgiving and thinking that the cost-benefit of doing it even a few days sooner would have been huge, holiday be damned.
Let's not forget the CDC advisory council had to meet after the FDA and there was a day in between because of their day jobs. The army should have been putting them on flights to DC that night.
FWIW, I signed up on One Day Sooner, and I would have done a challenge trial for probably any of the candidates. But COVID was a unique event*, and I don't think there's nearly as much utility for challenge trials in most cases.
*at least until we all get bird flu in 2025!
I still don't think the case for challenge trials for COVID was as strong as its proponents made it out to be; remember, the first vaccines were approved over three years faster than the fastest ever to date, not to mention that even had they been approved mid-summer, the manufacturing delays wouldn't have allowed access any more quickly. But I don't dispute that even moving things up 1-2 weeks wherever possible would likely have saved lives, and it's probably something than someone has calculated.
Nobody cares how hard you try. We could have approved on animal models+human immunogenicity back in April just like we do for vaccines to some really nasty things.
Manufacturing delays were largely downstream of approval issues and unwillingness to pay the amount the vaccines were worth. And now we've dropped the capacity. We should be able to respond to a pandemic in 60 days.
“Manufacturing delays were largely downstream of approval issues…”
The Wall Street Journal had a really good long-form article back in December 2020, about the entire vaccine effort at Pfizer. They were producing doses of vaccine as fast as they could starting in the spring of 2020 when they first knew they had a good vaccine:
“Starting in March, Mr. McDermott spent $500 million to buy and design equipment, more than double what he budgeted. Pfizer retrofitted an Andover, Mass., plant to make the mRNA, and assigned a St. Louis factory to make the raw materials for the shots. Numerous times it returned to suppliers or contractors to increase orders.
“Instead of making mRNA in giant steel tanks, which would add months, Pfizer used disposable bags. Each fits at least 500,000 doses.
“In April, the company began buying machines the size of a single-car garage that play a crucial role in production.”
https://www.wsj.com/articles/how-pfizer-delivered-a-covid-vaccine-in-record-time-crazy-deadlines-a-pushy-ceo-11607740483?mod=article_inline
I don’t think we’re years away from AI being effective in this realm. The progress in even the last six months in using LLMs for genomics has been astonishing. The piece talks about something much further along, using LLMs for automatic redaction - that seems completely feasible today.
That's genomics, *not drug discovery* or clinical trial design/output. The distance from genomics even to drug discovery is vast!
I think AI will *increase* costs rather than decrease costs. But for good reasons - AI will be better at helping us design new molecules for new targets but that will exacerbate costs by giving us more and better specialty drugs for new conditions.
You would be shocked at how expensive white-collar mistakes are at Pharma companies. Documentation inaccuracies in Clinical Trial Protocols mandate protocol re-approval, and can be the source of hundreds of thousands to millions of direct and indirect costs to trial sponsors.
see
https://pubmed.ncbi.nlm.nih.gov/30227022/#:~:text=The%20median%20direct%20cost%20to,for%20a%20phase%20III%20protocol.
+ This. There is a growing body of Academic/Industry research on clinical trials + AI (ie Jimeng Sun @ UIUC, James Zhou @ Stanford, Hoifung Poon @ Microsoft) exploring a myriad of topics.
A lot of the application research takes an expensive, technically challenging problem, and attempt to formalize it by creating a dataset one can benchmark against, and proposing a non-naïve solution using state of the art ML methods.
The problems approached are much broader than just drug discovery, but also operational challenges like patient to trial matching, publication evidence synthesis, statistical analysis.
Also, look at roles at top-20 pharma companies. Most of these organizations have their own technology and AI teams that are trying to cut operational costs using LLMs and traditional ML methods.
As someone who doesn't live in America, one thing I've always wondered about is why there's so much focus on what the American regulatory regime does.
Like, can't these global drug companies go to Vietnam or Nigeria or Indonesia (or even Japan) and work with the government there to get fast, cheap, effective trials in place with millions of people?
I know the answer is probably something like: the drug companies only care about selling to Americans for exorbitant amounts of money and aren't interested if they can't do that so that's why they don't do that but still..... There's this weird default assumption that America should pay all new drugs that the entire planet gets... Even among people that probably wouldn't agree with that if you asked them explicitly.
Pharma companies do run trials in the countries you listed. But they still need to meet the FDA's standard of evidence to get approval in the US (same thing in the EU). And one underrated factor here is that the US has a much lower level of corruption than other places... so the regulatory agencies will raise a skeptical eyebrow toward data that comes from a place where those norms are not as strong.
My understanding is that there's also the concern that if drug trials are done only in poorer countries with less oversight, less institutional experience with quality clinical trials, etc. that the data would be suspect (if not outright fake). There's also the PR problem for what happens if a rich country pharma company tests out a drug that turns out to be low quality with major side effects in a poorer country.
There have been some instances where drug trials in low or middle income countries used an inferior control arm— ethics requires you to use standard of care in the control arm, but the control arm was not standard of care. This can make the investigational drug look more effective than it is.
That’s not to say this can’t happen in rich counties, but it’s less likely to happen in rich countries with strong regulators
Most trials are global in nature and include sites across many countries. You wouldn't want to test a new drug in only 1 country because the results may not be applicable to different populations. The countries you listed can be part of a global trial, but can't be the totality.
The biggest issue with running trials in poorer countries is they don''t have the infrastructure to manage a clinical trial. You need staff to handle the extra paperwork, equipment for different tests, ability to store and ship biological samples etc. Also, in certain diseases it matters what standard of care patients have access to. For example, cancer drugs show proof of concept in later lines of therapy. A patient in Nigeria whose disease wasn't resistant to the standard of care drugs available in the US and EU wouldn't be helpful to determine the efficacy of a new drug. Patient retention can also be a bigger problem.
“ Like, can't these global drug companies go to Vietnam or Nigeria or Indonesia (or even Japan) and work with the government there to get fast, cheap, effective trials in place with millions of people?”
Pharma executives, eager to get their bonuses and cash in on their stock options, would conduct regulatory arbitrage and push fake trials in whatever country would look the other way.
I can’t imagine that peddling ineffective and/or unsafe drugs would boost their share prices.
The people behind aducanumab would beg to differ
I don’t think the aducanumab example supports the point you’re citing it for; more the opposite. The share price of Biogen suffered in the wake of the failure of that product and the CEO who championed it lost his job.
They came astonishingly close to robbing the entire US healthcare system blind with a drug that was known to not be effective. It was a very, very high expected value bet.
An example of government failure.
Yes, like BronxZooCobra originally said, there's a regulatory arbitrage. That usually implies a government failure of some kind. And the result is that in some not terribly uncommon cases "peddling ineffective and/or unsafe drugs would boost their share prices".
There was no regulatory arbitrage, the FDA ignored its own expert panel who said not to approve the drug.
Right like Boeing making planes that crash didn’t help their share price.
My brother is a 737 pilot for a major US carrier. His take is that the crashes were mostly a pilot-training issue, and there's a reason they happened in poorer countries.
For the specific MAX crashes, yes, that's true. But they're evidence that the larger culture at Boeing had changed from being run by competent airplane engineers to the financial executives. You can either focus on planes being cheap or being safe, and Boeing switched their priority to disastrous effect.
Nope - the final report of the second crash was the plane almost instantly entered an unrecoverable state. The aerodynamic forces on elevator were too high to manually trim.
American and French investigators both released comments stating that the flight crew did not follow correct emergency procedures. The Ethiopians apparently disagreed.
https://en.wikipedia.org/wiki/Ethiopian_Airlines_Flight_302#Final_report
I flew twice on a Boeing plane last month, successfully landing. It doesn’t seem to me that they’re building unsafe aircraft.
Hundreds of dead people would disagree.
It would definitely boost their share prices until they were caught, and the individual executives would have left before they were caught.
That can be, and probably already is, handled through employment agreements that claw back options when malfeasance is discovered.
They'd still be liable. Skilling was prosecuted even after leaving Enron and Sarbanes-Oxley has only strengthen the regulations since.
Because other rich countries won't pay more, and love free-riding off American pharmaceutical R&D and revenue.
Or do the trials wherever and let FDA accept them for the US.
There are two parts to your question that are usually tied together, but are worth considering separately:
1. Why do we focus on how clinical trials are run in the US?
2. Why do we focus on how drugs are approved in the US?
There is no rule that says -- and in fact it is often not the case -- that drugs must have solely (or even predominantly) US trials to be approved in the US. International trials are under-done because of risk aversion among trial designers, as Ruxandra discusses a bit in her companion post (https://www.writingruxandrabio.com/p/on-clinical-trial-abundance). But, for the ones that are (or for the minority US component of a multinational trial), it's worth focusing on the US because it can be improved.
The second question -- can't we focus on getting drugs approved elsewhere? -- is basically at the mercy of the economic fact that the US does half the world's healthcare spending, and an even greater fraction of the world's spending on on-patent drugs (meaning, their first ~ten years on the market, before competition drives their cost down to close to the cost of production). Speaking as someone who does financial modeling for drug developers, the accepted wisdom is to make sure your drug is a good investment if you consider just the odds of US approval and the size of the US market, then consider the secondary markets (probably starting with Europe) as nice add-ons.
*Should* this be the accepted wisdom? Well, it is. The US is the richest country in the world (by any reasonable metric) and by a large margin, and so on some level it can afford to (in the same way that it can bankroll a substantial portion of work against AIDS in sub-Saharan Africa). On another level, I don't agree with the US-pays-everyone-freerides system, but it's not like I have any personal leverage to convincing Europe to pay more for their healthcare, so what's the use? We work with what we have.
This is why we should be extracting tribute from other countries
We do. That's what skill-based immigration is.
Trials are commonly conducted in Eastern Europe for the reasons you state. Typically, costs per subject are much lower.
The challenge with other countries is that there are real, practical barriers to conducting good studies there. For example, it can be very hard to import all of the drugs, devices, and supplies needed. It’s common for a trial to use a device that isn’t approved in all countries and it may not be possible to get it through customs in a place like Vietnam. Another challenge is having consistent results of blood tests (labs). when different laboratories use different equipment, the results can’t always be compared on an apples-to-apples basis. Because of this, many drug trials use central labs where they ship all blood samples to a single lab. But that is also dependent on shipping the samples in roughly the same time. That’s very hard to do from somewhere like Nigeria. And, clinical study investigators aren’t made equal. In countries that don’t do many trials, enrollment or even the results could be impaired by having newbie investigators.
Given all this, there’s a minimum set of capabilities needed to do a lot of high-quality drug trials. there are not many countries with lower cost and these capabilities. None of this has to do with regulations.
They already go to other countries to do trials and studies...
Additionally, some countries in the EU and of course Japan are actually stricter in some ways related to drug development and clinical trial management than the US is. however there are trials happening in other countries, and in order to meet the FDA standards related to the clinical studies and new drug application they need to do certain things to sell in the US market.
I feel like I don’t understand the trade offs argued for here enough to know if this is persuasive. The whole article has a bit of a Chesterton fence problem.
I don’t understand the specifics of the rules we have well enough to say if I’m for the reforms listed here.
The tradeoffs are speed/efficacy in approving new drugs vs. higher risk of unforeseen adverse effects, including possibly extremely bad ones. Remember thalidomide?
People tend to have a status quo bias and generally judge sins of commission much more harshly than sins of omission. So, "this drug could have been approved but out of an abundance of caution it wasn't, so people with this illness will continue to suffer/die for lack of effective treatment" = it's just the way things are. "The FDA approved this drug and it unexpectedly caused fatal heart attacks in 0.1% of users" = this is an outrage, the FDA are criminals (and their director probably deserves to be shot, at least according to some edgelords on X).
To clarify, I am explaining, not excusing. It's likely that the status quo is TOO risk averse. Scott Alexander has made this point on his Substack, ACX.
There was a heartbreaking series of posts on ACX by a patient with advanced cancer (forget which type) who would write about the difficulties and hassles of enrolling in clinical trials. The patient has since died of cancer, leaving behind his wife and young child. He had become a bit of a celebrity in ACX world and there was an outpouring of sympathy at his death.
I understand that you're explaining, not excusing, but your comment doesn't really come across as "hey Ruxandra and Willy? here's some advice, you might not have realized it, but you're likely to run into resistance regarding..."
It's certainly true that risk aversion and omission/commission bias and medical history that was rewritten for political ends are headwinds -- but do you think they are reasons that Ruxandra and Willy should *give up on* the clinical trials abundance agenda? Or are you trying to point out specific things that they should be more aware of / cautious about than they currently are being?
Making eight proposals without a good précis of how the current process works is bad writing
The suggestions are reasonable but it’s not clear how much difference they would make. There’s also some hand-waving (AI !!!)
This piece was too dry to be interesting and too jargony to teach me much. It might have been salvaged by making the proposals more than bullet points and focusing on the most important three or four.
I liked it!
Have you ever not liked a guest post?
Not a big fan of the "Three Day Presidential Election" or "How to Save Local Journalism"
Take a look at Ruxandra's own substack today. That piece might have been a better fit here: https://www.writingruxandrabio.com/p/on-clinical-trial-abundance.
Thanks for highlighting. There's more detail here (and at least they actually note that RECOVERY was on dexamethasone, which is a repurposed drug with a known safety profile), but it still fails to address the problem of discovering unknown safety issues with new chemical matter and/or new biological pathways. None of that is something that you can hand wave away with AI. You need to test the drugs for long term effects, and that simply takes time!
Maybe this was misinterpreted. We do not think all trials can be like RECOVERY, but RECOVERY itself would have taken much longer if the urgency spurred by COVID wouldn't have allowed them to bypass a lot of cumbersome administrative requirements, which is something Martin Landray talks about. So, obviously, a trial for a new drug with unknown safety profile won't take as little as RECOVERY, but that does not mean there are no efficiency gains to be made!
To what extent is this just a question of risk aversion? Would it be worth accepting a higher risk of long-term adverse effects to get more new drugs more quickly? Do we even have enough data to understand the trade offs?
The medical profession seems to default to standard of care rather than engaging tradeoffs. A young doctor friend of mine urged me, 47 years old, no family history of cancer before 80, to have a colonoscopy. I said it sounded really unpleasant and I don’t want to shell out deductibles to fast and have a camera shoved up my ass. I asked “what are the odds I have colon cancer.”. She had no idea. I consulted Dr. Google and pegged the odds that not getting s colonoscopy kills me at about 1 in 5000. I’ll take the risk! I really wish the medical profession were more willing to discuss trade offs and to tweak the care different patients get based upon their risk tolerance.
>the odds that not getting s colonoscopy kills me at about 1 in 5000. I’ll take the risk!
I think you haven't accounted for your potentially getting colon cancer (God forbid) but it's found later, without a colonoscopy, after symptoms like blood in your stool. You might still live, but after a very unpleasant year or two of radiation targeting your GI, chemotherapy, in the worst case major bowel surgery, etc.
If you have to have major surgery to address late-stage colon cancer, there's a nonzero chance that you have to live with a colostomy bag for the rest of your life. You know what that is? Want to consult Dr. Google on that?
The point of the colonoscopy is to find & treat it early. You can 'survive' later-stage cancer but at a terrible QOL. Not sure you've properly evaluated all of the facts here
Eh a friend got her first colonoscopy at 47 and they found a tumor the size of a grapefruit.
I’ll also add death isn’t the only bad thing prevented. They also remove polyps while they are in there which, while not cancer, can turn into cancer. Getting a polyp removed vs. having to have your colon removed and spending the rest of your life with a colostomy bag. Sure they successfully treated you with surgery, radiation and chemo and you didn’t die. But having colostomy bag for the rest of your life is still going to suck.
A colonoscopy with a twilight sedative, money and time aside (mine was public-funded), is as they once said “a nothingburger”. And they chop off the polyps before they potentially become a problem.
Plus aren’t rates of colon cancer going up?
My husband had a routine colonoscopy recently. The day before the procedure he couldn't eat any solid foods and he had to drink a gallon of tasteless saline to flush out his colon. The doctor had told him frankly: "This will make you poop constantly all day, make sure you spend the whole day near a toilet." And so he did.
I 100% agreed that he needed a colonoscopy, and I'll have one too when my time comes (I'm 4 years younger than my husband), and a colonoscopy >>>>>> painful death of colon cancer. But let's not sugarcoat it. Getting a colonoscopy sucks ass.
I've had four colonoscopies. I agree that colonoscopies are not that bad, the worst part is having to fast for 24 hours before the procedure.
I applaud you making a reasoned choice that fits with your priorities.
I wouldn't -- and haven't -- make the same choice. The procedure itself is a nothingburger. The day before is unpleasant to be sure but a pretty small price to pay for peace of mind, imo.
Therein lies our difference. I am resigned to the fact that it’s all going to fall apart. My vitality will wither. There will come a point where I hope I have the balls to blow my brains out as the alternative is worse. Good health is evanescent. Sacrificing an evening to fasting and an afternoon of recovery and a few grand in copays for “peace of mind” is poor value. My life is barely worth living, I don’t want to sacrifice happiness to extend it.
The implication is that the admin requirements are what lead to the long timelines and large expense. It is absolutely a factor (I agree with your points on this, though my knowledge on this is peripheral so I take your word for it). But the key driver of long timelines is (a) examining efficacy in new systems and (b) examining safety effects. Neither of these things can be done without time, and these get longer and more complicated the more we learn about human biology. AI as of now can't solve this -- it's not even capable of finding new biological targets (the majority of papers published around this are proof-of-concept studies around repurposed kinase targets).
Details and questions aside I am very happy to see this issue addressed. Given how fundamental good health is to well-being (selection bias in day to day life hides just how many people’s lives are degraded by poor health outcomes) I’ve often wondered why we don’t allocate significantly more public money to medical research but of course it’s all important to get the most bang for your buck too.
Since it's slightly relevant here, I just want to note how lucky we are in Indiana to have Todd Young, the nation's third-best republican Senator.
Great stuff! I like bringing in experts to discuss issues like this rather than just relying on us generalists (i.e., strong on opinions, not so much on facts and expertise).
I look forward to a well-led FDA and HHS vigorously pursuing these important reforms starting on Jan. 20, 2029.
One thing I'm curious about, for people who have experience doing clinical research: in this essay, Scott Alexander tries to do a simple study (comparing the results of an existing psychological screening questionnaire to a psychologist's expert judgment, in the process of seeing the psychologist's existing patients; no plausible mechanism by which participants can be harmed.) He describes a horrific amount of pointless obstacles raised by his Institutional Review Board, which explain why no one does simple and harmless studies. https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
My question is, is this at all representative? Are truly pointless bureaucratic obstacles a major impediment to clinical research? If so, that seems like low-hanging fruit to remove -- and as long as it persists, it needlessly hinders evidence-based medicine and denies patients access to more effective diagnostics and treatments.
One factor that distinguishes Scott's story from -- I'm sorry -- professional clinical research, is that he was basically limited to the procedures at [specific hospital], and is S.O.L. if they don't prioritize their office of clinical research (including IRB); a developer-sponsor would pick from across the country (ideally, the world, but often they stop at "the country" for a variety of good and not-good reasons) for centers whose research offices do...better than Scott's hospital's did.
It is not *easy* to move the football forward ten yards, but it is not quite as hard as you might think if your one attempt was to run straight into the middle of the defensive line. (The opposing team is the terrible brokenness of the world, not any coalition of people who are actually evil. Usually.)
This certainly looks reasonable to an outsider. I wonder if it woud not be worthwhile asking, why, if these things make senses, the relevant agencies have not already done them? I suspect it is that like other agencies they do not have Prime Directive that regulations must pass a cost benefit test.
Dropping down a level of specificity, I was surprised by the absence of international cross approval among agencies and reference to best regulatory practice in other countries.
It's because the FDA has an incredibly risk adverse internal culture that ignores Americans dying while they do their jobs. They literally delayed the biological insulin approval by months just for the PR issues if it did in fact cause problems. Also the kidney guy who approved no drugs and would sandbag on the beltway. After he retired thousands of American lives were saved by the drugs he hadn't acted on.
Phenomenal guest essay. Especially the additional policy memo's that go into more detail. We need the groups like PhRMA and the Biotechnology Innovation Organization to get on board and actually use their lobbying power for good stuff like this. There are lot of operational efficiency opportunities in these memo's.
On the other side, there seems to be a huge disinformation push from some figures in the media world that don't trust any clinical trial data, or efficacy information until they of course get sick or their family needs a drug. Additionally, I think that too much venture funding is in the emerging biotech space, and because a rejected trial may affect stock prices they don't necessarily want to take risks on multiple assets that aren't billion dollar drugs. It's all a very complicated process, either way making trials easier to administer and manage would go a long way.
I still don't think Generative AI will solve every problem, since you still need wet labs, dry labs and of course actual clinical data. We've solved a lot of the big problems, but novel therapies, oral treatments, toxicity levels and of course trials that look into long term adverse events are still needed in my opinion, however Generative AI will and is currently a gamechanger on the discovery and R&D side.
Thanks for bringing in Ruxandra Teplo as a guest post. I enjoy her writing and I enjoy this topic.
I wholeheartedly support RCTs in medicine and elsewhere. However, I think the article does not put sufficient focus on a fundamental RCT limitation: time-to-outcome. Example: "Some clinical trials are unavoidably expensive and difficult: for example, trials with new biologics have to be manufactured in small batches, administered with physician supervision to watch for unknown adverse events, and involve collection and analysis of many biological samples per patient." There is no mention that in many diseases (most cancers, for example) you have to wait many years to observe outcomes. The incremental innovations described here, while very welcome, do not change this fact. I think this ought to be a significant part of the discussion. At the very least, it should be pointed out that the proposals will not make a tangible impact in some important disease areas. I am pretty sure the authors are aware of this, but not everyone is, and it should be made clearer, in my opinion.
For context, I am an AI scientist working on clinical applications. I worked on cancer AI for 13 years before switching to infectious diseases literally because the cancer RCTs take too long. I worked on this trial that took 6 years to execute (and yielded negative result):
https://www.annalsofoncology.org/article/S0923-7534(19)60360-1/fulltext
A big benefit here would be a national death registry and a law allowing pharma companies to access this data for any trial participants. List to follow-up and censuring are a challenge for cancer trials specifically.
The complex systems podcast did an episode about drug development that discussed some of the issues that make trials so expensive: https://www.complexsystemspodcast.com/episodes/drug-development-ross-rheingans-yoo/
Some things that stood out to me were: the cost of patient recruitment due to overly specific criteria and the FDA's insistence on American trials. Both things that got sidestepped in some of the successful COVID era trials.
Correction (as the podcast guest): I don't think that that FDA insists on American trials! I think that this is a misconception by over-cautious trial sponsors, as Ruxandra discusses in her companion post: https://www.writingruxandrabio.com/p/on-clinical-trial-abundance
I agree that Covid-era therapeutics trials, with one exception, fell into the buckets "under-recruited and failed" or "happened outside the US" (edit: or both), and that that experience is absolutely worth learning from.