The Problem in Science and Medicine is Not at the Tails but In the Middle of the Curve

Vinay Prasad has an excellent post on Sensible Medicine about scientific fraud. He wrote about a recent Freakonomics podcast about bad science.

Outright fraud always gets big attention. Whether it’s scientists making up data or doctors putting in too many stents. And, yes, of course, these are terrible events.

VInay is right, though. Outright fraud is not the main problem—neither with science nor healthcare.

Let’s do science first. Vinay writes about the lack of good science. He cites studies that are done to publish papers. For instance, observational studies that have no chance of sorting out causality. Every week I shake my head at these studies. I ask to myself: why would someone do this?

I would add to Vinay’s list the problem of motivated science. In high school, we learn that the purpose of science is to ask questions about nature. The problem with much of medical evidence is that its purpose is to show that something (usually profitable) works.

The middle of the curve of healthcare science therefore is full of experiments designed to show that a profitable drug or device passes some regulatory bar. As such, the experiments are flawed. Not fraudulent but not truly scientific.

For instance, say you want to know if a device that clips a leaky tricuspid valve helps patients. It is a good question.

But the experiment that we get tests the device by doing an invasive procedure in one group vs meds in the other. No placebo. Then the experiment measures a subjective endpoint. Every single person who reads this experiment knows that it is essentially useless. Even the scientists who designed and did the trial know it is flawed. And there it sits in the NEJM. Soon to be used at FDA and then in marketing materials.

Medical science is chock full of these studies. They are almost never fraudulent. But far too many of these studies are broken beyond utility.

The middle of the curve is the issue in medical science.

---

And it’s the same in medicine. A doctor in some rural county is busted for placing stents when there is essentially no disease. This is criminal. It makes news.

But it’s an outlier.

The real issue is everyday overuse, over-diagnosis and medicalization.

Coronary calcium scans on elderly people already on statin drugs. Nuclear stress tests on people who wear diapers. Mammography or colonoscopy in older people with dementia. Starting heart failure drugs—proven effective in ambulatory young outpatients—in frail older in-patients.

Low-value harmful care sits in plain site. It has been normalized—perhaps because it is profitable. Perhaps because doctors don’t pay enough attention to critical appraisal. Perhaps because doctors just want to get along.

Yet no one wants to do news stories about the mundane everyday horse-hockey that happens in the middle of the curve. It’s too hard.

One of my goals here, and at Sensible Medicine, and Cardiology Trials, and on This Week in Cardiology is to spread the value of critical appraisal. When you actually look hard at the evidence it makes you more humble about what to expect from our interventions.

I am no nihilist. Medicine is a great field. I love it. Our core problem however is not outlier bad behavior but low-value care that sits in plain sight.