I'm late to the party (and a nobody for that matter) but I disagree with your second point (ie treatment unblindedness leads to
differential HFHs). If your argument is that physicians in the trial favor the intervention and then are more likely to admit trial
patients in the control group (biasing the treatment effect towards the treatment), one could posit a similar but reversed argument:
physicians in the trial truly want patients to improve and then are more likely to admit trial patients in the *treatment* group because they want to be sure the treatment works (biasing the treatment effect
against the treatment).
Unblindedness in trials for medical devices is almost unattainable, as any physician committed enough can identify the patient's arm with imaging or checking device transmissions. For instance, a committed
physician in the GUIDE-HF trial (blinded) could have easily identified treated patients using the device transmission log (or lack thereof).
I do agree with your first point: without a sham intervention/true placebo, comparative QoL effects from a device trial are an upper bound of the true QoL effects.
Actually, I had a very similar experience with The Lancet a few years ago.
I had been invited to review a paper on a novel cardiac imaging method using CT. Of course, I wrote a review, pointing out a significant flaw in their study design that likely rendered the results unreliable or at best, highly questionable. AFter I had submitted my review, The Lancet invited me to write an editorial on that study. I wrote a (I thought pretty good) editorial that pointed out the flaws and thus of course turned out quite critical. The Lancet thanked me and also notified me that the editorial would be sent to the authors for their information. Shortly thereafter, The Lancet suddenly wrote me that my editorial was too harsh, and... instead of at least asking me to consider revising it, the journal ... DISINVITED me! Some days later, I received a phone call from one of the co-authors whom I have known since many years, with a soft apology about how this all panned out, putting all the responsibility on The Lancet.
I should add that the studied imaging method was, and is, key technology of a startup company, with several authors being either employees or shareholders of.
I always thought highly of The Lancet. Well, not anymore. I have not read it since.
This story sadly confirms that this may not have been an exception.
Sounds like a lot of minimally effective drugs. The new Alzheimer's drug is in that category. Thank God for the internet. Sometimes it takes a while for Science to win, but it will. Someone is sure to pitch a fit about this and there will be lawsuits. SUE.
I've no real knowledge of how to conduct studies and have no health expertise, however on reading your article I would be most unhappy; what's happened to integrity these days, it seems to have gone AWOL. These are people who should know better, how do they sleep at night profiting from misleading work?
Thank you for emphasizing this important point. Mentioning conflict of interest is an important point, but under- emphasized. Quote Karl Marx: tell a lie once, it is a lie - but tell it 100 times, it becomes the truth. <uch of science is telling lies repeatedly, as you have pointed ou.
It beggars belief that an unblinded study would have anything other than the hardest endpoints as the primary efficacy outcome (all cause mortality; and, if short of that, at least CV death….maybe not so much 3 point mace in a trial about worsening HF).
It also beggars belief in the post ORBITA world to have a device trial without sham control.
Yet here we are. That the trial authors acknowledge yet persevere with these glaring design flaws; and that the major journals publish such studies without a stern editorial note, belies the quid pro quo that sustains the medical science distribution paradigm of our time. It not only invites, but requires, skepticism at a minimum…..and likely cynicism among all but the most naive.
For the authors and editorialists to recognize the limitations of the study design while still drawing the conclusions they did, is analogous to “declarations” of conflicts of interest. It gives an “air” of propriety, but does nothing to mitigate the inherent impropriety.
John -- Thank you for this insightful critique. My trust in medical journals and their editorial practices is long gone. Your analysis is another nail in a coffin already sealed with countless nails.
I read medical studies as if they are infomercials, and am rarely disappointed.
Seeing the deep contamination of medical science by commercial interests, I wonder where lies the remedy. Maybe start a journal entitled "The Journal of Non-Biased Medicine"?
How do we press the journals to have some conviction? I am very frustrated and I am losing faith in the discipline I have practiced and pushed for over 20 years. It makes me wander what else I have believed and or been pushed to do because of “best practice or guidelines”. I really makes me sick.
As usual, great analysis. Gone are the days when when a researcher could ask a simple question, like "does Digitalis (originally derived from the Foxglove plant and costing pennies) have a role in heart failure treatment?" As I see it, there are 2 major flaws underpinning modern device and drug trials. One, is that they are almost always funded by the companies that stand to gain. The second is that they are executed by physicians who also stand to gain through increasing their procedural volumes, attracting more research and advisory money and advancing their academic careers. Unless the editors of major medical journals and the FDA decide to be more actively critical of research designs and conclusions, we are lost.
Well, what can I say? It ´s really sad the way the world is doing science. Evrything ends with: "Oh the p value is <0.05, then it´s ok".
The vast universe of MDs don´t know about basic statistics and science investigation methodology. It reminds me the TRILUMINATE trial where, among other things, the triclip device was changed in the middle of the randomization process and this information is in A FIGURE in the supplementary app of the NEJM!!!!!!
My first reaction when I read is not to trust anything. But I recognize that we are snipers. Journals and guidelines are mainly driven by industry and for us It´s very difficult to stand on the other side.
Thanks for posting. Unfortunately your findings are not a one off. Doug Altman published “The scandal of poor medical research”, in the BMJ in 1994. A follow up publication in 2014, “Sill a scandal” was published in the BMJ. A recent study was published in the BMJ Open stating that about 74% of USA guideline authors have relationships with industry and about 40% of them failed to disclose their relationships even when required to do so. We need to do what you are doing which is to critically evaluate the trials that are published.
Dr. Mandrola,
I'm late to the party (and a nobody for that matter) but I disagree with your second point (ie treatment unblindedness leads to
differential HFHs). If your argument is that physicians in the trial favor the intervention and then are more likely to admit trial
patients in the control group (biasing the treatment effect towards the treatment), one could posit a similar but reversed argument:
physicians in the trial truly want patients to improve and then are more likely to admit trial patients in the *treatment* group because they want to be sure the treatment works (biasing the treatment effect
against the treatment).
Unblindedness in trials for medical devices is almost unattainable, as any physician committed enough can identify the patient's arm with imaging or checking device transmissions. For instance, a committed
physician in the GUIDE-HF trial (blinded) could have easily identified treated patients using the device transmission log (or lack thereof).
I do agree with your first point: without a sham intervention/true placebo, comparative QoL effects from a device trial are an upper bound of the true QoL effects.
I disagree. You're much more likely to hear "This guy is safe to send home, we're monitoring him anyways", which is what I suspect happened.
Alot of our HF admissions are not about how the patient is now, but how he's going to be tomorrow morning.
Thank you John. So well-written.
Actually, I had a very similar experience with The Lancet a few years ago.
I had been invited to review a paper on a novel cardiac imaging method using CT. Of course, I wrote a review, pointing out a significant flaw in their study design that likely rendered the results unreliable or at best, highly questionable. AFter I had submitted my review, The Lancet invited me to write an editorial on that study. I wrote a (I thought pretty good) editorial that pointed out the flaws and thus of course turned out quite critical. The Lancet thanked me and also notified me that the editorial would be sent to the authors for their information. Shortly thereafter, The Lancet suddenly wrote me that my editorial was too harsh, and... instead of at least asking me to consider revising it, the journal ... DISINVITED me! Some days later, I received a phone call from one of the co-authors whom I have known since many years, with a soft apology about how this all panned out, putting all the responsibility on The Lancet.
I should add that the studied imaging method was, and is, key technology of a startup company, with several authors being either employees or shareholders of.
I always thought highly of The Lancet. Well, not anymore. I have not read it since.
This story sadly confirms that this may not have been an exception.
Sounds like a lot of minimally effective drugs. The new Alzheimer's drug is in that category. Thank God for the internet. Sometimes it takes a while for Science to win, but it will. Someone is sure to pitch a fit about this and there will be lawsuits. SUE.
I've no real knowledge of how to conduct studies and have no health expertise, however on reading your article I would be most unhappy; what's happened to integrity these days, it seems to have gone AWOL. These are people who should know better, how do they sleep at night profiting from misleading work?
Thank you for emphasizing this important point. Mentioning conflict of interest is an important point, but under- emphasized. Quote Karl Marx: tell a lie once, it is a lie - but tell it 100 times, it becomes the truth. <uch of science is telling lies repeatedly, as you have pointed ou.
.
C'mon Guys ...
The People That Thought
That Taking The Vaccine
Was A Good Idea
- Couldn't Be Responsible For Their Own Health.
Given What We Know Of Them
How Could They Be ?
.
Dr. Mandrola,
I think this is just the tip of an enormous iceberg. I've written about the iceberg at drwohlgemuth.com.
I would really like to know what you think. andreww@Maine.edu.
Once again, 100% agreement! This kind of behavior must be be unveiled
It beggars belief that an unblinded study would have anything other than the hardest endpoints as the primary efficacy outcome (all cause mortality; and, if short of that, at least CV death….maybe not so much 3 point mace in a trial about worsening HF).
It also beggars belief in the post ORBITA world to have a device trial without sham control.
Yet here we are. That the trial authors acknowledge yet persevere with these glaring design flaws; and that the major journals publish such studies without a stern editorial note, belies the quid pro quo that sustains the medical science distribution paradigm of our time. It not only invites, but requires, skepticism at a minimum…..and likely cynicism among all but the most naive.
For the authors and editorialists to recognize the limitations of the study design while still drawing the conclusions they did, is analogous to “declarations” of conflicts of interest. It gives an “air” of propriety, but does nothing to mitigate the inherent impropriety.
Follow the money.
And the money follows
John -- Thank you for this insightful critique. My trust in medical journals and their editorial practices is long gone. Your analysis is another nail in a coffin already sealed with countless nails.
I read medical studies as if they are infomercials, and am rarely disappointed.
Seeing the deep contamination of medical science by commercial interests, I wonder where lies the remedy. Maybe start a journal entitled "The Journal of Non-Biased Medicine"?
How do we press the journals to have some conviction? I am very frustrated and I am losing faith in the discipline I have practiced and pushed for over 20 years. It makes me wander what else I have believed and or been pushed to do because of “best practice or guidelines”. I really makes me sick.
As usual, great analysis. Gone are the days when when a researcher could ask a simple question, like "does Digitalis (originally derived from the Foxglove plant and costing pennies) have a role in heart failure treatment?" As I see it, there are 2 major flaws underpinning modern device and drug trials. One, is that they are almost always funded by the companies that stand to gain. The second is that they are executed by physicians who also stand to gain through increasing their procedural volumes, attracting more research and advisory money and advancing their academic careers. Unless the editors of major medical journals and the FDA decide to be more actively critical of research designs and conclusions, we are lost.
Well, what can I say? It ´s really sad the way the world is doing science. Evrything ends with: "Oh the p value is <0.05, then it´s ok".
The vast universe of MDs don´t know about basic statistics and science investigation methodology. It reminds me the TRILUMINATE trial where, among other things, the triclip device was changed in the middle of the randomization process and this information is in A FIGURE in the supplementary app of the NEJM!!!!!!
My first reaction when I read is not to trust anything. But I recognize that we are snipers. Journals and guidelines are mainly driven by industry and for us It´s very difficult to stand on the other side.
Thanks for posting. Unfortunately your findings are not a one off. Doug Altman published “The scandal of poor medical research”, in the BMJ in 1994. A follow up publication in 2014, “Sill a scandal” was published in the BMJ. A recent study was published in the BMJ Open stating that about 74% of USA guideline authors have relationships with industry and about 40% of them failed to disclose their relationships even when required to do so. We need to do what you are doing which is to critically evaluate the trials that are published.