Discussion about this post

User's avatar
Frank Harrell's avatar

You made some good points but let's stop and think about the composite outcome components issue. An overriding principle in my mind is to construct endpoints that reflect how patients feel or fare. Hospitalization is bad and should be counted but not as much as say MI or stroke. If you feel that non-HF hospitalization is bad but is not as bad as HF hospitalization then in an ordinal outcome rank non-HF hosp as less severe than HF hosp (and this less severe than MI and MI less severe than death). The end result of this is that we will be able to get a single expression of which treatment provides better patient outcomes, and will greatly reduce sample size requirements. More at https://hbiostat.org/endpoint

Expand full comment
JohnS's avatar

I can see how requiring the preregistration of the endpoints would dramatically reduce positive results.

Suppose we do a large RCT testing whether eating jellybeans causes cancer. The results are negative. But the lead scientist is convinced jellybeans cause cancer because he has seen it so many times in his practice. It occurs to him, maybe it’s a particular flavor that causes cancer. There are 20 flavors of jellybeans, so he looks carefully at the data. Sure enough, the green ones appear to cause cancer as the p value is under 5%.

Of course, with 20 flavors at least one of them is likely to show a false positive. Preregistering endpoints puts an end to this kind of nonsense. To clear this up maybe someone should try to replicate a few of those old studies.

Expand full comment
13 more comments...

No posts