In a recent Viewpoint piece in the Journal of the American Medical Association, Prasad et al. discuss some the challenges in turning medical practice away from established therapies. They begin by observing that in the ideal case, an old practice is replaced by a new one after the new practice is robustly evaluated in comparative trials. Yet there are also cases in which a practice must be abandoned not because there is something better, but because it is shown to be no better, or actually worse, than nothing.
As examples of the latter, they cite vertebroplasty — the injection of cement into a fractured bone for the treatment of osteoporotic fractures and hormone therapies and stenting, using drugs and artificial tubes to relieve a body’s constricted pathways, like a vein or artery — for the management of stable coronary artery disease. Both of these surgical practices were shown to be no better than sham procedures.
Although changing the practices of physicians is never easy, for the non-ideal cases of needing to abandon what has been considered an effective standard of care is especially difficult. Prasad et al. argue that the problem is part economic, part epistemic. They write:
There are thousands of clinical trials, but most deal with trivialities or efforts to buttress the sales of specific products. Given this conundrum, it is possible that some entire medical subspecialties are based on little evidence. Their disappearance probably would not harm patients and might help salvaged derailed health budgets. However, it is unlikely that specialists would support trials testing practices that constitute their main source of income. Instead, the research community performs studies of modest incremental value without even knowing whether the basic standards of care are appropriate.
In other words, although we have asked questions about these (now known to be) faulty treatments, we haven’t been asking the right questions. Prasad et al. go on to point out, as studies increasingly rely on surrogate end points and short-term outcomes, their results do not accurately reflect the clinically relevant outcomes. This limits both the credibility and the quality of the evidence, obscuring the inferential relationship between the trial and clinical practice.
Given the difficulties in changing established clinical practices, Prasad et al. urge that regulators should raise the standards for initial approval. The expert medical community should, in turn, insist on multiple large, well-designed trials before promoting a novel intervention to standard medical prac
tice.
But we can take the analysis one step further by emphasizing an old, and yet underappreciated, distinction between explanatory and pragmatic trials (cf. Schwarz and Lellouche 1967). The essential insight in this distinction is that there is more than one kind of clinical trial. An explanatory trial is designed to address questions about an intervention’s biological efficacy — for example, “How much does this drug shrink the size of a tumor?” A pragmatic trial, on the other hand, is designed to address questions about an intervention’s effectiveness in clinical practice — for example, “Does the use of this drug reduce overall patient mortality?” Both of these kinds of trials can be important for our medical understanding, but they will be important for different reasons.
Knowing the precise magnitude of a drug’s shrinking effect on tumors is valuable information about a drug’s efficacy, and is certainly relevant to its clinical import. But such knowledge is not sufficient for use in clinical practice. Suppose this drug also causes intense discomfort while ultimately having no impact on patient mortality. This is information about effectiveness, and it makes the contrast between explanatory and pragmatic trials clear. An explanatory trial with this drug, using a surrogate endpoint of tumor size, would make this drug appear promising. But a pragmatic trial, looking at long-term patient survival, would not.
Thus, a way to sharpen Prasad et al.’s point is to demand that effectiveness, rather than efficacy, ought to be the bar for an intervention’s regulatory approval and a pre-requisite for standard practice. Short-term studies with surrogate endpoints are still valuable studies. They are (typically) less expensive and provide helpful information about treatment effects while we are still exploring the potential of a new medical intervention. But a rigorous test is not necessarily the right test. When we want to know whether or not we should be using a treatment in clinical practice, then we require more than a demonstration of efficacy. We require a robust demonstration of effectiveness.
Spencer Hey, PhD
References.
Prasad, V., A. Cifu, J. Ioannidis (2012). “Reversals of Established Medical Practices: Evidence to Abandon Ship,” Journal of the American Medical Association, Vol. 307, pp.37-38.
Schwartz, D and J. Lellouch (1967). “Explanatory and Pragmatic Attitudes in Therapeutical Trials,” Journal of Chronic Disease, Vol. 20, pp.637-648.
