I think you make some good points but you kinda overplay your hand in a number of places -- and while I ultimately tend to agree with you it's for the opposite reason.
1) Good point about surgeons being regulated by licenscing boards etc...but the fact that surgeons use anaesthetics or implant devices approved by the FDA is kinda besides the point (lots of variation in surgical technique besides the drugs and devices). So kinda weird to make it the title.
2) Is it really true that the effects of drugs are harder to measure of do we just hold them to a higher standard?
I mean, given that surgery is regulated as you describe if it is so much easier to determine its efficacy then how can it be that often it doesn't work that well?
More specifically, most drugs actually also so something pretty clear cut (put you to sleep, reduce cholesterol, kill bacteria, inhibit virus replication, reduce pain etc). They all do something clear cut if you include descriptions like: inhibit the blah protein or bind to the blah receptor.
Most of the expense and difficulty getting most drugs approved isn't in showing some efficicacy against **some** endpoint but showing that the drug isn't too harmful and is net beneficial (not phrased this way but that's essentially what's behind debates about using intermediate endpoints like cholesterol rather than reduction in heart attacks). There are plenty of surgeries that we've found out would fail those standards.
---
Ultimately, while I believe the FDA imposes far too high a barrier on approving new medications (just require it to what it claims to do at the rate and side effects claimed not that it improve on existing meds) I think it's justifed for medications but not surgery exactly because it is soo much harder to figure out what surgeries are worthwhile.
There are just too many variables and the initial attempts are often failures. Transplants were death sentences ... until they weren't. Drugs are a single item that can be evaluated in it's complete form before approval. Surgery would never develop vital new techniques if it had to pass some general test of side effects vs benefits to be approved because surgeons would never learn until approval and absent training the method would languish.
Luckily, you are correct that the risks with surgery are lower. People are much less likely to undergo surgery, particularly a novel one, unless they have no other option. They might decide to take thalidomide just because they feel a bit nauseous.
I think we basically agree, you just touch on a deeper epistemological point that I didn't deal with in this post: what is the most principled way to make decisions about new treatments?
I think in general surgery is more straightforward to reason about than drugs. But, as you point out, you can pick examples of drugs that clearly work (antibiotics) and surgeries that have ambiguous efficacy. In a broader sense, you can bucket new treatments into two classes:
1. Minimizing downside is preferable (vaccines for healthy people, cosmetic surgery)
2. Maximizing upside is preferable (deadly diseases with no good treatments, transplants)
The FDA grew up during the era of mass-market blockbusters so was designed to deal with class #1-adjacent treatments. We shouldn't have the same approach for both classes, but (in my view) that doesn't mean we scrap the FDA - it has an important function. Accelerated approval etc. tries to deal with the dichotomy of risk-preference but probably doesn't go far enough. I think there are conditions (e.g. n-of-1 rare disease) where there are convincing arguments that you should be able to bring a drug to market based on the judgement of reasonable experts and some preclinical tests.
It seems to me that selecting one of the buckets is itself a problem. The goal should always be to simply maximize the total benefit. Now that can get a bit difficult when you need to balance consumer confidence in the drug approval process but -- at least in the usual situation -- the FDA should at least set out a specific formula. This risk of this level of unpredicted side effects is allowed if approval adds such and such many QALYs to the country.
But, for all I agree that the FDA in some form needs to stay I'm much more cynical than you. Indeed, the **reason** we need the FDA tells us that it's going to be pretty dangerous/misguided. In the modern internet age doctors can look up drug information so if everyone was rational we could just reduce the FDA to acting as a repository of pre-registered studies. Unfortunately, people are too easy to excite and have trouble balancing risk well.
Unfortunately, there is every reason to believe the same issues infect the FDA deciscion making. I mean, I don't think it's fair to say the FDA is doing 1 given it's off-label prescribing rules. Indeed, something is deeply inconsistent about requiring a medication to show efficacy, rather than merely safety, (and indeed efficacy over existing treatments) but then turn around and say it's perfectly legal to use that drug approved as an unusual anti-psychotic for everything from cigarette cravings to mood disorders.
Rather, I think what explains the FDA deciscions is a kind of emotional balance between being afraid people will be hurt and wanting to see people helped by both the political and scientific parts of the agency. They are all good people trying hard, but that's why I strongly believe they need to do some economics work to actually estimate the costs imposed by having such high barriers to approval and approach the issue more quantatatively. Otherwise there is too strong a push to avoid being the person who okays the next thalidomide.
But ok, it's subject to strong political constraints so maybe
Agree. I think we should have more regulatory flexibility and lower premarket evidence standards for conditions with high unmet need (similar to what we've seen with Exondys and Elevidys in DMD). Just because there are domains where the FDA's approach doesn't work all that well doesn't means we should throw the baby out with the bathwater, in my view.
I don't think we should blame the FDA for all the problems with clinical trials either. Helping patients find promising trials and improving trial logistics/access are important issues that we can ameliorate without having to wait around for the FDA.
"Helping patients find promising trials and improving trial logistics/access are important issues that we can ameliorate without having to wait around for the FDA."
One of the big problems is time and uncertainty: drug makers often don't entirely know what the FDA is going to ask of them. And they don't know when the FDA will react to submissions. Those long time lags were highlighted during COVID but are a huge problem in general.
IMO it should move straight to some kind of conditional approval, but apparently the FDA is making Merus, the maker, do a phase 3 trial. People are out here dying! Not making petosemtamab, which is a borderline miracle drug for R/M HNSCSC, available, seems insane.
Maxwell was arguing that surgery is unregulated, if you just ignore all the hoops and regulations.
Your debate is converging on the conclusions that the FDA must exist and that it also must become less bloated. Unrestrained regulatory agencies will always overreach - it’s in their nature. The onus is on us (the royal “Us”) to recognize this truism and build in constant trimming and leaning-down, lest we end up with unnavigable frigates.
Also, the main regulation of surgery in day-to-day practice in via the peer review committees of hospital medical staffs. It’s a slow and painful process, but it generally bends toward quality.
I'm new to what seems like an ongoing debate between the two of you, but it seems you are arguing past each other. From reading Maxwell's linked post I don't think they are arguing that removing the FDA will result in no scams/harm. I think they are arguing that while scams/harm exists and will continue to exist at a potentially higher rate without the FDA, the benefits provided by an unregulated pharmaceutical market outweigh those costs.
There is the heavily regulated world we currently live in, where it is extremely hard/expensive to create therapies, and many potentially good therapies never make it to market or do only after many decades of unnecessary suffering and death have occurred. There is a hypothetical unregulated world where some people fall victim to marketing, scammers, and sometimes dangerous products. In the regulated world, we still have some scammers and dangerous products, it is just less common (in principal). In the unregulated world we would still have good therapies not making it to market, it just happens less (in principal).
I don't think the point of contention is around either of these world descriptions (which is what it seems both of you are arguing about) or any hypothetical in-between world (e.g., regulated, but less so), it is instead around exactly how much harm is done (e.g., lost QALY) by the regulators/bureaucracies vs how much harm is done by scammers and profit seekers. In other words, you both agree harm occurs in both the regulated world and the unregulated world, you just disagree on how those two volumes of harm compare.
A point that I think favor's Maxwell here is your example of the supplement market. There are certainly a lot of low quality products out there, and even a handful of actively harmful ones, but I don't think human society is losing a significant amount of QALY because some Vitamin D tablets are garbage, and I think making it hard/expensive to produce/acquire Vitamin D would likely negatively impact QALY significantly. The same goes for the supplement industry as a whole, D is just meant as an illustrative example here.
Surgery has a natural control - in many places cosmetic procedures like liposuction, breast implant, etc aren't covered under general medical surgery, and they're usually less regulated. There's also issues with marketing dangerous procedures like the Brazilian Butt Lift (BBL).
You could compare post-op infection rates between the more regulated and less regulated surgery sub-fields.
"Until they were stamped out by regulators in the early 20th century, patent medicine hucksters sold ineffective, and sometimes literally poisonous, nostrums to desperate patients. We still use “snake oil” today as shorthand from a scam product."
Except that these are unrelated points. Snake oil salesmen were scammers because their products didn't contain snake oil. *actual* snake oil is very high in omega-3 fatty acids, its benefits are similar to fish oil, and fish oil has had an FDA-approved version since 2019.
Similar, any argument that post-thalidomide expansions of FDA authority has to contend with the fact the the FDA rejected thalidomide. Yes, you can argue it was too close a call, but the thalidomide birth defects happened in Europe, not here.
You can very consistently argue that requiring that products contain what they claim, as the original creation of the FDA did, was good, without thinking that requiring prescriptions, or requiring proof of efficacy, or requiring proof of not-actually-defined-but-left-to-FDA-discretion levels of safety are good,
Your argument neglects that what is important when evaluating drugs is the context of use. It's like saying it's fine to bring a grenade into a kindergarten because grenades are useful in warfare.
The problem with snake oil salesmen (a term which references a class of scammers selling all sorts of "patent medicines") is that they made all sorts of claims that were unsupported by the evidence. It doesn't follow that because fish oil is FDA approved to treat elevated triglycerides it's also ok to promote snake oil for treating cancer.
On thalidomide, it doesn't matter that the FDA rejected it. The thalidomide crisis was the trigger that spurred governments to implement existing proposals to evaluate the efficacy of a drug as well as safety (i.e. the risk-benefit profile) when issuing marketing approvals, if not for thalidomide it would likely have been some other drug. Thalidomide being used in leprosy or myeloma has little bearing on the decision to reject the marketing authorisation for less severe conditions like morning sickness.
I understand all that, but lumping so many scams together under the term "snake oil salesman" muddies the reasoning too much, I think. The *original* rationale for the FDA was only a small subset of that, focusing on things being that they say they are. It's possible to disagree with "Is it on balance better for the FDA to have the powers it was granted in year 19XX?" with a different answer for each value of XX. For myself, I think the FDA was very net positive pre-thalidomide and debatable at best since, but most likely actively harmful.
I think you make some good points but you kinda overplay your hand in a number of places -- and while I ultimately tend to agree with you it's for the opposite reason.
1) Good point about surgeons being regulated by licenscing boards etc...but the fact that surgeons use anaesthetics or implant devices approved by the FDA is kinda besides the point (lots of variation in surgical technique besides the drugs and devices). So kinda weird to make it the title.
2) Is it really true that the effects of drugs are harder to measure of do we just hold them to a higher standard?
I mean, given that surgery is regulated as you describe if it is so much easier to determine its efficacy then how can it be that often it doesn't work that well?
More specifically, most drugs actually also so something pretty clear cut (put you to sleep, reduce cholesterol, kill bacteria, inhibit virus replication, reduce pain etc). They all do something clear cut if you include descriptions like: inhibit the blah protein or bind to the blah receptor.
Most of the expense and difficulty getting most drugs approved isn't in showing some efficicacy against **some** endpoint but showing that the drug isn't too harmful and is net beneficial (not phrased this way but that's essentially what's behind debates about using intermediate endpoints like cholesterol rather than reduction in heart attacks). There are plenty of surgeries that we've found out would fail those standards.
---
Ultimately, while I believe the FDA imposes far too high a barrier on approving new medications (just require it to what it claims to do at the rate and side effects claimed not that it improve on existing meds) I think it's justifed for medications but not surgery exactly because it is soo much harder to figure out what surgeries are worthwhile.
There are just too many variables and the initial attempts are often failures. Transplants were death sentences ... until they weren't. Drugs are a single item that can be evaluated in it's complete form before approval. Surgery would never develop vital new techniques if it had to pass some general test of side effects vs benefits to be approved because surgeons would never learn until approval and absent training the method would languish.
Luckily, you are correct that the risks with surgery are lower. People are much less likely to undergo surgery, particularly a novel one, unless they have no other option. They might decide to take thalidomide just because they feel a bit nauseous.
I think we basically agree, you just touch on a deeper epistemological point that I didn't deal with in this post: what is the most principled way to make decisions about new treatments?
I think in general surgery is more straightforward to reason about than drugs. But, as you point out, you can pick examples of drugs that clearly work (antibiotics) and surgeries that have ambiguous efficacy. In a broader sense, you can bucket new treatments into two classes:
1. Minimizing downside is preferable (vaccines for healthy people, cosmetic surgery)
2. Maximizing upside is preferable (deadly diseases with no good treatments, transplants)
The FDA grew up during the era of mass-market blockbusters so was designed to deal with class #1-adjacent treatments. We shouldn't have the same approach for both classes, but (in my view) that doesn't mean we scrap the FDA - it has an important function. Accelerated approval etc. tries to deal with the dichotomy of risk-preference but probably doesn't go far enough. I think there are conditions (e.g. n-of-1 rare disease) where there are convincing arguments that you should be able to bring a drug to market based on the judgement of reasonable experts and some preclinical tests.
It seems to me that selecting one of the buckets is itself a problem. The goal should always be to simply maximize the total benefit. Now that can get a bit difficult when you need to balance consumer confidence in the drug approval process but -- at least in the usual situation -- the FDA should at least set out a specific formula. This risk of this level of unpredicted side effects is allowed if approval adds such and such many QALYs to the country.
But, for all I agree that the FDA in some form needs to stay I'm much more cynical than you. Indeed, the **reason** we need the FDA tells us that it's going to be pretty dangerous/misguided. In the modern internet age doctors can look up drug information so if everyone was rational we could just reduce the FDA to acting as a repository of pre-registered studies. Unfortunately, people are too easy to excite and have trouble balancing risk well.
Unfortunately, there is every reason to believe the same issues infect the FDA deciscion making. I mean, I don't think it's fair to say the FDA is doing 1 given it's off-label prescribing rules. Indeed, something is deeply inconsistent about requiring a medication to show efficacy, rather than merely safety, (and indeed efficacy over existing treatments) but then turn around and say it's perfectly legal to use that drug approved as an unusual anti-psychotic for everything from cigarette cravings to mood disorders.
Rather, I think what explains the FDA deciscions is a kind of emotional balance between being afraid people will be hurt and wanting to see people helped by both the political and scientific parts of the agency. They are all good people trying hard, but that's why I strongly believe they need to do some economics work to actually estimate the costs imposed by having such high barriers to approval and approach the issue more quantatatively. Otherwise there is too strong a push to avoid being the person who okays the next thalidomide.
But ok, it's subject to strong political constraints so maybe
I don't disagree with you. The 2 buckets was an oversimplification to make the point. As i've said elsewhere, I like this Bayesian framework suggested by Andrew Lo: https://www.sciencedirect.com/science/article/abs/pii/S0304407618302380
Drug development for fatal diseases works poorly with the FDA: https://jakeseliger.com/2024/01/29/the-dead-and-dying-at-the-gates-of-oncology-clinical-trials/
Agree. I think we should have more regulatory flexibility and lower premarket evidence standards for conditions with high unmet need (similar to what we've seen with Exondys and Elevidys in DMD). Just because there are domains where the FDA's approach doesn't work all that well doesn't means we should throw the baby out with the bathwater, in my view.
I don't think we should blame the FDA for all the problems with clinical trials either. Helping patients find promising trials and improving trial logistics/access are important issues that we can ameliorate without having to wait around for the FDA.
"Helping patients find promising trials and improving trial logistics/access are important issues that we can ameliorate without having to wait around for the FDA."
True, and the topic of another essay! https://jakeseliger.com/2023/10/31/puzzles-about-oncology-and-clinical-trials/
One of the big problems is time and uncertainty: drug makers often don't entirely know what the FDA is going to ask of them. And they don't know when the FDA will react to submissions. Those long time lags were highlighted during COVID but are a huge problem in general.
In terms of regulatory flexibility, MCLA-158 / petosemtamab, the cancer drug I'm on now, is a good example: https://beta.clinicaltrials.gov/study/NCT03526835?intr=MCLA-158&rank=1. The phase 1 data is amazing: https://www.onclive.com/view/petosemtamab-generates-early-clinical-activity-and-tolerability-in-hnscc. Recurrent/metastatic HNSCC is a death sentence. Petosemtamab is effective against it. Phase 2 data hasn't been published yet but looks really good.
IMO it should move straight to some kind of conditional approval, but apparently the FDA is making Merus, the maker, do a phase 3 trial. People are out here dying! Not making petosemtamab, which is a borderline miracle drug for R/M HNSCSC, available, seems insane.
Maxwell was arguing that surgery is unregulated, if you just ignore all the hoops and regulations.
Your debate is converging on the conclusions that the FDA must exist and that it also must become less bloated. Unrestrained regulatory agencies will always overreach - it’s in their nature. The onus is on us (the royal “Us”) to recognize this truism and build in constant trimming and leaning-down, lest we end up with unnavigable frigates.
Also, the main regulation of surgery in day-to-day practice in via the peer review committees of hospital medical staffs. It’s a slow and painful process, but it generally bends toward quality.
I'm new to what seems like an ongoing debate between the two of you, but it seems you are arguing past each other. From reading Maxwell's linked post I don't think they are arguing that removing the FDA will result in no scams/harm. I think they are arguing that while scams/harm exists and will continue to exist at a potentially higher rate without the FDA, the benefits provided by an unregulated pharmaceutical market outweigh those costs.
There is the heavily regulated world we currently live in, where it is extremely hard/expensive to create therapies, and many potentially good therapies never make it to market or do only after many decades of unnecessary suffering and death have occurred. There is a hypothetical unregulated world where some people fall victim to marketing, scammers, and sometimes dangerous products. In the regulated world, we still have some scammers and dangerous products, it is just less common (in principal). In the unregulated world we would still have good therapies not making it to market, it just happens less (in principal).
I don't think the point of contention is around either of these world descriptions (which is what it seems both of you are arguing about) or any hypothetical in-between world (e.g., regulated, but less so), it is instead around exactly how much harm is done (e.g., lost QALY) by the regulators/bureaucracies vs how much harm is done by scammers and profit seekers. In other words, you both agree harm occurs in both the regulated world and the unregulated world, you just disagree on how those two volumes of harm compare.
A point that I think favor's Maxwell here is your example of the supplement market. There are certainly a lot of low quality products out there, and even a handful of actively harmful ones, but I don't think human society is losing a significant amount of QALY because some Vitamin D tablets are garbage, and I think making it hard/expensive to produce/acquire Vitamin D would likely negatively impact QALY significantly. The same goes for the supplement industry as a whole, D is just meant as an illustrative example here.
Surgery has a natural control - in many places cosmetic procedures like liposuction, breast implant, etc aren't covered under general medical surgery, and they're usually less regulated. There's also issues with marketing dangerous procedures like the Brazilian Butt Lift (BBL).
You could compare post-op infection rates between the more regulated and less regulated surgery sub-fields.
"Until they were stamped out by regulators in the early 20th century, patent medicine hucksters sold ineffective, and sometimes literally poisonous, nostrums to desperate patients. We still use “snake oil” today as shorthand from a scam product."
Except that these are unrelated points. Snake oil salesmen were scammers because their products didn't contain snake oil. *actual* snake oil is very high in omega-3 fatty acids, its benefits are similar to fish oil, and fish oil has had an FDA-approved version since 2019.
Similar, any argument that post-thalidomide expansions of FDA authority has to contend with the fact the the FDA rejected thalidomide. Yes, you can argue it was too close a call, but the thalidomide birth defects happened in Europe, not here.
You can very consistently argue that requiring that products contain what they claim, as the original creation of the FDA did, was good, without thinking that requiring prescriptions, or requiring proof of efficacy, or requiring proof of not-actually-defined-but-left-to-FDA-discretion levels of safety are good,
Your argument neglects that what is important when evaluating drugs is the context of use. It's like saying it's fine to bring a grenade into a kindergarten because grenades are useful in warfare.
The problem with snake oil salesmen (a term which references a class of scammers selling all sorts of "patent medicines") is that they made all sorts of claims that were unsupported by the evidence. It doesn't follow that because fish oil is FDA approved to treat elevated triglycerides it's also ok to promote snake oil for treating cancer.
On thalidomide, it doesn't matter that the FDA rejected it. The thalidomide crisis was the trigger that spurred governments to implement existing proposals to evaluate the efficacy of a drug as well as safety (i.e. the risk-benefit profile) when issuing marketing approvals, if not for thalidomide it would likely have been some other drug. Thalidomide being used in leprosy or myeloma has little bearing on the decision to reject the marketing authorisation for less severe conditions like morning sickness.
I understand all that, but lumping so many scams together under the term "snake oil salesman" muddies the reasoning too much, I think. The *original* rationale for the FDA was only a small subset of that, focusing on things being that they say they are. It's possible to disagree with "Is it on balance better for the FDA to have the powers it was granted in year 19XX?" with a different answer for each value of XX. For myself, I think the FDA was very net positive pre-thalidomide and debatable at best since, but most likely actively harmful.