I am not cynical but these two studies...
The treatment of patients with chronic heart failure has come a long way. But new treatments may not represent substantial gains--for patients, that is
Before I tell you about two recent (mind-boggling) studies, I want to say thank you for the support. It’s shocking how many have subscribed and supported this newsletter.
This will be the first of a two-part series regarding two curious studies.
I have built a 30-minute lecture on critical appraisal of medical evidence. I give the talk in seven chapters.
The first chapter urges learners to consider what a study is for. Was the study done to answer an important question? Or was its purpose marketing of a drug or device?
One of my colleagues, a former academic, who is now in industry, tells me I shouldn’t have this chapter. He says it is cynical and if I say a study was done for marketing it might insult the researchers.
Let me tell the story of these studies and you be the judge.
Study 1: PARAGLIDE-HF
The Journal of the American College of Cardiology published last month a study called PARAGLIDE-HF. It compared the use of the drug sacubitril/valsartan (Entresto) vs valsartan in patients with an ejection fraction of ≥ 40% after a worsening heart failure event.
Background: Sacubitril/valsartan gained approval for use in patients with heart failure and a reduced ejection fraction based on one trial—called PARADIGM-HF. Controversy persists over the approval because most new drugs require two regulatory trials, and because features of PARADIGM-HF favored the new agent.
Heart failure that occurs in people with a preserved ejection fraction is more common. To enter this larger market, the makers of sacubitril/valsartan funded a trial called PARAGON-HF, which compared sacubitril/valsartan to valsartan.
In PARAGON-HF, the new drug reduced a composite primary outcome of heart failure events and death from cardiovascular causes but the reduction failed to meet statistical significance. Here was the conclusion in the NEJM:
Yet the drug still gained regulatory approval for all patients with heart failure. How, you ask?
Three ways: proponents showed FDA subgroups within the main trial that seemed to benefit more from sacubitril/valsartan; they proposed different ways to analyze events, and they published meta-analyses where PARAGON-HF was mixed with other positive trials in patients with HF and a reduced ejection fraction.
I disagreed strongly with this approval. But I do not have a say at FDA.
Yet uptake of the expensive drug for patients with the most common kind of heart failure has been slow.
Enter PARAGLIDE-HF. On the surface, the idea of doing another comparison of sacubitril/valsartan to valsartan, is a positive. Replication is good.
Here is the problem. The Novartis-led PARAGLIDE-HF trial randomized only 466 patients. (PARAGON-HF randomized 10x more.) If the much larger trial could not show a statistically significant benefit in outcomes, how could a trial 1/10th the size tell us much?
Well, the authors of the smaller trial did not choose clinical outcomes, such as heart failure or death, as the primary endpoint. Instead they measured a biomarker call BNP (brain natriuretic peptide). BNP is a common blood test used by doctors to help sort out the fluid status of patients. But BNP is a non-specific marker. And no patient would ever care what her BNP was—if not told by doctors.
And sure enough, PARAGLIDE-HF found a 15% relative risk reduction of this biomarker in the sacubitril/valsartan arm. This modest difference barely met statistical significance. The p-value came in at 0.049.
PARAGLIDE-HF authors also collected clinical events such as heart failure hospitalizations and death but the trial was way too small (too few events) to say anything about these signals. Our jargon is “the trial was not powered for clinical events.”
One other important finding was the 73% higher rate of low blood pressure in the sacubitril/valsartan arm.
The first time I read this study I was confused. What was the point?
It seemed foolish to randomize 10-fold less than the previous trial and then measure a non-specific biomarker. What we wanted to know was whether the new combination drug (sacubitril and valsartan) was better than generic valsartan at reducing serious events.
Then my friend and super-sleuth electrophysiologist Bogdan Enache alerted me to the reason. He explained PARAGLIDE-HF to me in a Tweet
Bogdan quote-tweeted a link to a pooled analysis of PARAGLIDE-HF and PARAGON-HF published in the European Heart Journal.
The authors of the pooled analysis wrote that PARAGLIDE-HF was not powered to answer clinical outcomes, but they could improve the power by combining patients in PARAGLIDE-HF with patients in PARAGON-HF who were enrolled within 30 days of a heart failure event. (Early enrollment of patients after a HF event was a minor twist of PARAGLIDE-HF.)
Remember — PARAGLIDE-HF and the pooled analysis were published on the same day but PARAGON-HF was published almost 4 years ago.
Of course. Now it makes sense.
PARAGLIDE-HF and it’s 466 patients had no chance of showing any signal of clinical outcomes. Bogdan’s point was that it was used to seed the larger (and non-significant) PARAGON-HF trial.
Cardiologist and regulatory expert Sanjay Kaul had the key Tweet (HHF = hospitalization due to heart failure).
PARAGLIDE-HF had added 440 patients to the old PARAGON-HF trial. The effect size as measured by the hazard ratio was still 0.87 or a 13% reduction in the primary endpoint of heart failure or CV death.
But, now, because of these extra 440 patients the confidence intervals of the pooled analysis are ever-so-slightly narrower and the upper bound went from 1.01 to 0.99. The p-value went from 0.06 to 0.04.
Boom! A “positive” study for sacubitril/valsartan—published in EHJ, a high impact factor journal.
Conclusion
Let me start with a screenshot from the PARAGLIDE-HF clinicaltrials.gov site.
This is something, isn’t it?
The heart failure with preserved ejection fraction market is huge. Do you think PARAGLIDE-HF and the pooled analysis will be cited in guidelines, and in advertisements?
So…I ask: What was the 466-patient PARAGLIDE-HF trial for?
Any master of the obvious would answer that It was to add a few more patients to a non-significant regulatory trial published almost four years ago.
What other conclusion can one make when proponents publish an (also positive) pooled analysis on the same day?
But. But. it seems unfair to combine trials this far apart. Don’t do a small seeding trial with an unimportant primary endpoint simply to add patients to an old trial. If you are confident in your drug, do another proper trial.
What’s more, the modest effect size of only 13% is not different. As Dr. Kaul implied in his Tweet, the needle has not moved.
Yet the expensive combination drug now has added two “positive” studies to its resume.
Gosh. How would any regular doctor know any of this?
In part 2, coming soon, I will tell the story of a similar trial, also in the heart failure space.
In the comments, I am open-minded to not being cynical about these things. Feel free to persuade me.
Not cynical, empirical and parsimonious. William of Ockham would approve.
Wise and thoughtful analysis... sometimes it is ok to be cynical...